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Abstract 

This  research  presents  a  new  approach  to  improving  the  performance  of  a  macro  platmer : 
selective  reuse,  hi  macro  planning,  reuse  can  result  in  poorer  performance  than  when  planning 
witfi  only  primitive  operators,  a  phenomenon  fliat  has  been  called  the  utility  problem.  The 
utility  problem  arises  because  the  benefits  of  reuse  are  outweighed  by  the  cost  of  retrieving  a 
maao  to  reuse  and  the  cost  of  searching  through  the  larger  search  space  caused  by  considering 
reuse  candidates.  Selective  reuse  contains  the  expansion  of  file  search  space  by  limiting  the 
number  of  reuse  candidates  considered  and  limits  file  search  required  by  considering  only 
fiiose  reuse  candidates  fiiat  entail  no  additional  work.  Previously,  performance  improvement 
in  a  macro  planner  has  been  possible  only  by  selective  learning.  Unlike  selective  learning, 
selective  reuse  never  overlooks  a  learning  opportunity  that  might  have  value  in  future  problem 
solving.  This  research  developed  a  polynomial-order  retrieval  method  which  reduces  the 
cost  of  retrieving  a  reuse  candidate  likely  to  save  search.  The  retrieval  method  requires  time 
that  varies,  at  worst,  linearly  with  file  increasing  size  of  the  library  of  macros.  A  macro 
planner  (HINGE)  was  implemented  to  explore  selective  reuse.  Because  the  HINGE  planning 
model  can  insert  operators  anywhere  in  a  plan  and  does  not  introduce  arbitrary  ordering 
constraints,  HINGE  can  solve  more  problems  efficiently  through  reuse  than  classical  macro 
planning  models.  To  enhance  file  probability  of  beneficial  reuse,  decreasingly-powerfiil  reuse 
candidates  are  iteratively  sought  using  a  unique,  hierarchically-structured  search  mefiiod  which 
is  supported  by  several  problem  decomposition  techniques  and  by  the  planning  model  used. 
Performance  results  with  HINGE  are  ccmsistrat  wifii  the  idea  that  selective  reuse  can  contain 
file  utility  problem  without  file  need  for  selective  learning. 
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SEARCHING  FOR  PLANS  USING 


A  HIERARCHY  OF  LEARNED  MACROS 
AND  SEI^CnVE  REUSE 


I.  Introduction 

A  plan  is  a  set  of  actions  taken  in  some  order  to  achieve  desired  goals,  and  planning  is 
the  process  of  finding  a  plan.  Planning  is  drought  before  action,  decision  before  execution. 
Planning  is  a  fundamental,  essential  activity  of  all  intelligent  entities  because  widiout  it,  goals 
may  not  be  achievable  or  resources  may  be  wasted.  As  intelligent  entities,  humans  are  able 
to  plan  quite  well,  although  humans  often  malm  mistakes,  especially  as  the  complexity  of  the 
plan  increases. 

Under  certain  additions,  it  is  possible  to  write  a  computer  program  diat  plans  without 
making  mistakes.  Figure  1  shows  a  block  diagram  of  a  basic  planning  system.  The  inputs  to 
the  planning  system  define  a  planning  problem  and  consist  of  the  initial  state,  die  desired  goals, 
and  a  set  of  operators  for  changing  state  by  virtue  of  die  actions  they  represent.  A  planning 
system  works  by  searching  for  an  operator  sequence  which  transforms  the  initial  state  inm  a 
state  in  which  the  problem  goals  are  true.  This  operator  sequence  constitutes  a  plan.^ 

It  is  desirable  for  a  planning  system  to  always  find  a  plan  whenever  one  exists.  To 
guarantee  diis  property,  however,  exhaustive  search  is  normally  required  because  chosen 
seardi  directions  may  prove  to  be  fruitless.  Because  exhaustive  search  algoridims  require 
time  that  is  an  exponential  function  of  the  length  of  die  path  searched,  planning  systems  are 
often  slow  when  a  long  plan  is  required. 

^  An  operator  sequence  is  not  necessarily  required.  Chapter  n  describes  a  model  of  planning  that  allows  a 
plan  to  consist  of  a  set  of  operators  and  a  set  of  ordering  constraints  between  operators. 
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Initial  State  — 

Prot3iem  Qoals 

Operators  — 

Figure  1.  A  single  planning  system  that  searches  using  provided  operators. 

One  method  of  making  plamiing  systems  plan  faster-  is  to  use  more  powerful  operators, 
ones  that  achieve  more  goals.  Plans  made  using  these  operators  are  shorter  (and  thus  can 
be  found  more  quickly)  because  fewer  operators  are  required  to  achieve  the  same  number 
of  goals.  Using  powerful  operators  to  speed  planning  is  an  appealing  approach  because  it 
is  possible  to  leam  more  powerful  operators  from  plans  derived  during  problem  solving. 
Operators  learned  from  plans  are  called  macro  operators  (or  macros);  they  are  defined  by  the 
interface  characteristics  of  file  plan  they  represent  and  they  are  annotated  wifii  the  plan  so  fiiat 
fiiey  can  be  interpreted  by  file  user  of  a  planning  system. 

Figure  2  is  a  block  diagram  for  a  planning  system  that  learns  macros  and  reuses  them  to 
inqirove  its  planning  speed.  In  Figure  2,  the  original  operators  supplied  as  part  of  the  planning 
problem  are  called  ‘’primitive  operators”  to  distinguish  them  from  fiie  learned  macro  opera¬ 
tors.  Planning  systems  that  reuse  the  plans  that  macro  operators  represent  are  called  “macro 
plannm”  and  the  maaos  considered  by  the  planning  system  are  called  “reuse  candidates.” 

^la  diis  leseaich,  peifoniiaiice  refos  to  die  speed  widi  which  ^  planning  system  can  find  a  plan,  radier  than 
die  (piality  of  die  plan  produced. 


Figure  2.  A  planning  system  with  a  learning  element 
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Two  operalois  applicable 
at  each  choice  point 


Three  operators  applicabie  at  each  choice  point 


Figure  3.  Expansion  of  a  search  space  caused  by  considering  one  additional  choice  per 
decision. 

In  contrast,  simple  planners  like  the  one  shown  in  Figure  1  are  called  “generative  planners” 
because  they  generate  a  solution  using  only  the  primitive  operators  supplied. 

1.1  The  Utility  Problem 

An  empirical  observatitHi  from  using  macro  platuiing  systems  such  as  the  one  shown  in 
Figure  2  is  that  for  some  problems,  performance  is  worse  than  if  a  simpler  planning  system  Hlcft 
the  one  shown  Figure  1  had  been  used.  This  phraomenon  has  been  termed  the  utility  problem 
and  fliere  are  three  factms  ttiat  cause  it  First  there  is  overhead  associated  with  retrieving  a 
good  operator  from  die  library.  Second,  when  the  planning  system  considers  macro  operators 
in  additicm  to  the  primitive  operators,  die  larger  number  the  choices  available  expands  die 
seardi  space,  which  thus  requires  more  time  to  search  exhaustively.  For  example.  Figure  3 
shows  diat  the  additional  ctxisideration  of  1  operator  at  each  decision  point  in  search  increases 
the  size  of  the  search  space  and  die  number  of  possible  paths  in  it  dramatically.  Third,  inserting 
a  mao'o  operator  into  a  plan  can  inqiact  the  amount  of  future  planning  necessary  more  than 
insOTting  a  primitive  operator.  A  macro  operatm:  represents  a  large  step  in  the  search  space; 
if  it  is  a  step  in  the  wrcxig  direction,  then  die  planning  system  has  a  large  amount  of  future 
work  to  do  to  get  back  onto  a  padi  leading  to  a  goal  state.  These  three  factors  are  costs  or 
potential  costs  incurred  when  a  planning  system  reuses  learned  plans  (represented  as  macro 
tqierattH^).  Perfrxtnance  inqirovement  results  only  if  the  benefit  of  using  the  more  powerful 
macro  cqieratCH^  outweighs  the  costs  associated  with  die  utility  problem  as  identified  above. 
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Previous  research  has  addressed  die  utility  problem  by  limiting  the  overhead  of  retrieving 
a  good  macro  operadM*  or  by  strictly  limiting  fte  expansion  of  the  search  space  caused  by  the 
larger  number  of  operatm^  available  for  consideration.  To  limit  the  overhead  of  retrieving  a 
good  maao  operator,  one  ^proach  is  to  limit  the  size  of  die  library  by  limiting  the  number  of 
plans  that  are  learned.^  For  exan^ile,  Minttm  writes: 

“Searching  through  the  space  of  stored  plans  to  find  one  that  is  best-suited  to  the 
current  prtdilem  may  be  as  expensive  as  searching  through  the  original  search 
space.  This  leads  to  the  following  paradox:  as  the  system  gains  experience  it 
gradually  becomes  swamped  by  die  knowledge  it  has  acquired.  In  some  cases, 
performance  can  eventually  degrade  so  dramatically  that  the  system  operates 
even  more  poorly  than  a  non-learning  system.  ...  To  avoid  being  swamped  by 
too  many  macro-operators,  a  problem-solver  can  endeavor  to  retain  only  those 
maao-operators  that  are  most  useful”  (Minton,  1985:651) 

An  important  drawback  of  selective  learning  is  some  plan  that  would  be  useful  on  a  future 
problem  may  not  be  learned.  To  limit  the  expansion  of  die  searc..  space,  some  planning 
systems  ctHisider  only  one  maao  opaator  for  insertion  into  the  plan  it  is  developing.  Such 
planning  systems,  called  case-based  planners,  retrieve  a  maao  operator  which  represents  a 
plan  diat  solved  a  similar  problem  and  modify  the  opaator  in  some  way  to  make  it  solve  the 
current  problem.  An  inqxirtant  disadvantage  of  reusing  only  one  plan  is  diat  the  planning 
system  can  (mly  solve  problems  that  are  similar  to  problems  it  has  previously  encountaed. 


New  Ideas 

This  research  focuses  on  new  techniques  for  limiting  the  impact  of  the  utility  problem. 
The  utility  problem  arises  not  from  costs  alone,  but  because  costs  outweigh  the  benefits  of 
plan  reuse.  Thaefore,  I  have  treated  the  utility  problem  holistically  by  combining  mediods 
that  oicourage  successful  reuse  with  methods  for  limiting  die  costs  associated  widi  reuse,  hi 
particular,  I  have  designed  a  planning  model  that  explicitly  represents  features  of  desirable 
reuse  candidates  and  improves  die  flexibility  of  inserting  a  maao  opaator  into  a  developing 

^Limiting  die  number  of  macro  operaton  available  also  indiiecdy  limits  die  expansun  of  die  search  space 
because  fewer  alternatives  exist  at  choic'*  points  in  the  space  searched,  but  diis  is  not  die  primary  modvadan  fer 
selective  learning. 
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plan.  Most  previous  plan-reusing  planning  systems  have  been  based  on  a  planning  model  that 
limits  the  insertion  of  a  macro  operators  and  dtus  precludes  reuse  in  some  instances.  Also,  to 
encourage  successful  reuse,  I  have  developed  a  unique  search  ctHitrol  strategy  that  provides 
multiple  (^ptHtunities  for  reuse  dirough  problem  decomposititxi.  E>econq)osition  omtrols  die 
search  for  a  reuse  candidate  and  results  in  generative  planning  like  diat  implied  by  Figure  1 
when  no  acceptable  reuse  candidate  is  found. 

To  control  die  costs  associated  widi  the  utility  problem,  I  use  an  efficient,  info  i- 
rich  retrieval  method  which  limits  the  retrieval  overhead  without  die  need  for  selective  le^  mg. 
To  limit  the  search  space  expansion  and  search  itself,  my  ^proach  requires  that  only  (me  reuse 
candidate  can  be  (xinsidered  to  solve  a  problem  or  sub-problem  and  that  reuse  candidate  must 
be  on  a  solution  path  and  must  ratail  no  future  work.  I  call  diis  meduxl  selective  reuse,  and  it  is 
supported  by  die  ability  of  my  retrieval  meduxl  to  find  v^  specific  reuse  candidates.  Previous 
{qiproaches  have  encouraged  retrieval  of  a  reuse  candidate  diat  is  likely  to  be  (m  a  solution 
path  (M*  likely  to  entail  no  future  w(n‘k.  A  logical  extension  of  diese  tqiproaches,  selective  reuse 
omtains  die  utility  problem  better  than  other  mednxls.^  Selective  reuse  depends  (xi  having 
complete  infcxmatum  about  a  problem.  When  a  problem  is  spetdfied  inconqiletely,  seletdive 
reuse  must  be  relaxed  to  allow  reuse.  The  relaxed  fcxrm  of  seletdive  reuse  (xinsid^  only  a 
few  reuse  candidates  diat  are  likely  to  entail  no  future  wcxk.  Although  the  relaxed  fmn  of 
seletdive  reuse  is  mcxe  suscqitible  to  the  utility  prciblem  than  strud  selective  reuse,  it  ccxitains 
the  utility  problem  better  than  any  odier  reuse  policy,  given  the  amount  of  information  known 
about  the  problem. 

To  explcve  ^proadies  to  ctmtaining  die  utility  problem,  I  designed  and  implemented  a 
planning  system,  HINGE,  based  on  the  planning  model,  search  ccmtrol,  retrieval  mediod,  and 
policy  of  selective  reuse  I  develc^ied.  (Chapter  m,  IV).  By  virtue  of  these  attributes,  HINGE 
ctmtrols  die  utility  problem  extremely  weU. 

The  specific  contributuxis  made  by  this  researcdi  are: 

^  Selective  reuse  aAen  reaults  in  less  reuse  rad  dus,  stHoetimes,  worse  overall  pafonnance  ttaan  odier  mediods. 
but  die  aim  of  fliis  research  is  to  find  techniques  of  leductng  die  utility  problem. 
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1.  A  model  of  planning  and  reuse  that  e^qylicitly  represents  features  of  desirable  macro 
operators  and  supports  more  flexible  insertion  of  macro  operators  into  the  developing 
plan.  I  have  devel(^>e(l  a  planning  model  which  represents  distinguished  subsets  of 
die  outstanding  goals  as  abstract  c^attMrs  in  a  plan.  These  subsets  of  goals  are 
used  as  an  index  for  finding  macro  op^ators  which,  when  retrieved,  may  be  inserted 
anywhere  in  the  developing  plan.  In  axntrast  with  other  planning  models,  these  feahires 
potentially  iaq)rove  search  control  and  allow  reuse  diat  would  otherwise  be  precluded. 
(Sections  2.4. 3.2. 3.9). 

2.  A  unique  hierarchically-structured  search  control  mechanism  which  encourages  sub¬ 
problem  learning  and  improves  opportunities  for  reuse  while  ensuring  that  a  plan  will 
be  found  when  one  exists.  Unlike  ofiier  planning  systems.  HINGE  structures  its  goal- 
satisfacticxi  search  hierarchically  in  terms  of  the  number  of  goals;  this  structure  promotes 
both  learning  and  reuse  of  generalized  plans  or  sub-plans.  During  planning.  HINGE’S 
hierarchically-structured  search  omtrol  prefers  operatcus  that  solve  larger  pmtions  of 
the  problem  before  other  operators.  HINGE  can  also  decompose  a  set  of  goals  that  can¬ 
not  be  solved  simultaneously.  HINGE  does  not  diffo^tiate  betweoi  macro  operates 
learned  from  previous  planning  episodes  and  the  primitive  t^atra^  diat  are  supplied 
as  part  of  the  planning  problon  desoiption.  The  effect  of  fiiese  choices  during  plan¬ 
ning  is  to  facilitate  q)portunistic  reuse  of  whole-problem  solutions,  flexible  insertion  of 
subproblem  solutkxis  whoi  no  generalized  plan  solves  the  whole  problem,  and  graceful 
degradatuxi  to  planning  using  only  primitive  operators  whoi  reuse  is  not  possible.  If 
reuse  fails.  HINGE  solves  die  pr(d)lem  using  tmly  primitive  operaux’s  and  a  complete 
search  strategy;  HINGE  always  finds  a  solution  to  a  planning  problem  if  a  solutimi 
exists.  When  learning,  all  sub-parts  of  the  hierarchical  search  tree  cmrespcHiding  to 
sub-problems  that  can  benefit  from  reuse  are  learned  by  HINGE’S  learning  conqxxient 
(SectuMis  3.4, 3.6). 

3.  A  polynomial-order  method for  retrieving  appropriate  ( or  probably-appropriate)  macros 
from  a  library.  An  inqxntant  result  of  fliis  research  is  idoitification  of  a  polynomial 


6 


method  for  retrieving  macros  that  are  guaranteed  to  be  mi  a  path  to  a  solution  and  to 
entail  no  future  planning  work.  To  find  such  maaos.  the  retrieval  method  requires  com¬ 
plete  infmtnation  which  gmierally  exists  mily  at  die  beginning  of  a  planning  problem. 
Howevm.  even  if  the  problem  must  be  decomposed,  informatimi  often  exists  to  find 
macros  that  are  at  least  likely,  if  not  guaranteed,  to  be  on  a  path  to  a  solution.  A  related 
result  is  diat  this  same  retrieval  method  requires,  in  the  wcffst  case,  only  linear-order 
time  in  die  size  of  the  library  and  is  not  a  functimi  of  die  size  of  die  macros  in  die  library. 
(Section  4.4.3). 

4.  A  policy  of  selective  reuse  which  limits  the  number  of  macros  considered  and  considers 
only  those  macros  that  are  on  a  solution  path  and  entail  no  future  work.  This  specific 
search  control  policy  is  designed  to  improve  reuse  performance  by  stricUy  limiting  both 
die  size  of  the  search  space  and  die  search  required  in  it  Limiting  die  number  of  reuse 
candidates  considered  limits  die  expansion  of  the  search  space  caused  by  considering 
any  q;)eratims  in  additimi  to  primitive  (^imatms.  If  maoros  retrieved  are  guaranteed 
to  be  tn  a  sohitimi  padi,  then  it  is  not  necessary  to  cmisider  more  dian  one  of  them. 
Moreover,  if  macros  are  on  a  solutimi  path,  then  no  backtracking  is  required,  and 
if  macros  do  not  require  die  planner  to  do  any  future  wmk  on  their  behalf,  then  no 
seardi  is  required  to  make  sure  they  are  ^pUcable.  In  ccmtrast  if  a  macro  requires 
the  planner  to  achieve  additional  goals,  thoi  die  amount  of  additional  search  (planning 
work)  required  in  the  future  is  unknown  and  unconstrained.  >^thout  some  external 
source  of  knowledge,  die  inqiact  of  inserting  a  maao  into  a  plan  can  cmly  be  quantified 
when  no  additional  work  is  required  (and  then  die  quantity  is  zero).  Selective  reuse 
is  diereficHre  the  (xily  domain-indep^dent  mediod  of  limiting  the  future  wtxrk  oitailed 
by  inserting  a  macro  operator.  For  domain-independent  planning,  any  odier  reuse 
policy  re-introduces  die  utility  problem  by  expanding  die  search  space  and  requiring 
additional  search.  Selective  reuse  d^iends  on  having  a  conqilete  specification  of  the 
planning  problem.  If  die  planning  prc^lem  is  not  conqiletely  specified,  as  it  is  not  when 
solving  a  sub-problem  as  if  it  were  indqiendent  of  oth^  parts  of  the  problem,  then  a 
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relaxed  form  of  selective  reuse  often  gives  good  results  in  HINGE.  The  relaxed  form  of 
selective  roise  considers  only  diose  macros  diat  are  likely  to  be  on  a  solutitxi  path  and 
likely  to  entail  no  future  work  by  the  planner.  Both  fcnms  of  selective  reuse  ctHitain  the 
utility  problem  better  dian  alternative  policies.  (Sechcxis  4.3, 4.S). 

HINGE  is  die  first  macro  planner  that  axitains  the  utility  problem  by  selective  reuse, 
raditf  than  selective  teaming.  Furthermore,  HINGE  is  the  first  planner  diat  is  specifically 
designed  to  increase  the  qipcHlunities  for  reuse  through  hierarchical  problem  deconqiosituHi. 
Finally.  HINGE’S  retrieval  method  lowers  the  cost  of  reuse,  white  its  planning  model  facilitates 
search  control  and  increases  the  probability  of  effective  reuse. 

13  Dissertation  Organization 

This  dissertatirai  is  (vganized  into  six  main  ch^ters.  The  following  chtqiter  presents 
background  material  diat  serves  as  a  foundation  for  diis  research.  The  general  topics  of 
planning,  maao  teaming,  reuse,  and  the  utility  problem  are  described.  To  show  the  relatitmship 
of  this  research  widi  other  wtxk,  four  other  planning  systems  are  outlined  and  con^ared  to 
HINGE  in  terms  of  the  types  of  problems  solved,  the  flexibility  of  reuse,  and  management  of 
costs  associated  widi  the  utility  problem.  Chapter  m  describes  die  HINGE  planner  and  the 
features  of  its  model  that  promote  reuse.  HINGE’S  hierarchically-stmctured  search  method 
is  described  as  providing  mult4)te  opptvtunities  fOT  reuse,  trying  die  best  candidates  first 
HINGE’S  ability  to  flexibly  reuse  solutions  is  reiterated  and  shown  to  stem  firom  its  plan-space 
planning  model  Chi^ter  IV  analyzes  the  utility  problem  and  defines  two  methods  to  combat 
it  a  polyntunial  retrieval  method  and  die  policy  of  selective  reuse  fcH*  macros.  An  exanqite 
is  i^esented  to  suppwt  die  claim  diat  under  restrictive  ctMidititms.  it  is  possible  to  build  a 
polyncunial-order  macro  retrieval  procedure  that  retrieves  only  macros  that  are  guaranteed 
to  be  on  a  sohiticm  path.  Gutter  IV  also  shows,  by  example,  how  a  ncxi-selective  policy 

on  insertkm  of  reuse  candidates  expands  the  search  space  and  requires  additional  search, 

/ 

characteristic  symptoms  of  the  utility  problem.  Guitar  V  describes  empirical  results  whidi 
are  consistent  with  claims  made  about  the  develt^ied  retrieval  mediod  and  selective  reuse.  The 
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final  chapter  of  file  document  provides  conclusions,  reviews  specific  research  contributions, 
and  suggests  promising  avenues  fcv  future  work. 
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//.  Background 


2.1  Introduction 

Automated  planning  systems  can  be  applied  to  solve  many  real-world  problems,  but 
their  inefficiency  prevents  dieir  widespread  use.  One  method  of  improving  efficiency  is  to 
avoid  search  by  reusing  previously  derived  plans  as  much  as  possible.  This  strategy  does  not 
always  work,  however:  an  observatitHi  which  defines  the  utility  problem.  To  avoid  the  utility 
problem,  some  mechanism  must  be  employed  to  promote  plan  reuse  while  limiting  search.  To 
encourage  plan  reuse,  it  is  useful  to  identify  and  retrieve  good  reuse  candidates,  to  be  flexible 
when  inserting  a  selected  candidate,  and  to  provide  additional  opportunities  for  reuse  if  initial 
effcNTts  fail  This  research  improves  upon  previous  effuts  in  these  areas.  To  limit  search 
without  outside  infcMmatimi  to  guide  search,  die  number  of  reuse  candidates  considered  must 
be  limited  somehow.  Previous  methods  for  limiting  search  have  included  selective  learning 
of  plans  for  maao  planners  and  single-plan  reuse  with  modificatitm  ftn-  case-based  planners. 
The  restrictitm  of  reusing  a  single  plan  in  case-based  planning  is  an  exanple  of  a  nuH-e  general 
method  fiw  amtrolling  the  e2q)ansum  of  the  search  space:  selective  reuse.  Ch^tn^  HI  and  IV 
describe  a  macro  planning  system  (HINGE)  which  also  contains  the  search  space  expansicm 
by  selective  reuse.  This  ciuq)ter  presmts  die  the  background  informaticMi  that  facilitates 
discussions  in  duster  m  and  TV. 

As  an  overview,  diis  duster  begins  by  defining  terms  required  to  describe  HINGE  as 
weU  as  other  planners  and  then  uses  diese  terms  to  discuss  state-  and  plan-space  search  and 
related  issues.  Section  2.2.4  describes  a  non-detotninistic,  plan-space-searching  planning 
noodel  which  HINGE’s  planning  model  extoids.  Three  formal  prq)erties  diat  characterize 
HINGE  and  some  other  planners  are  defined.  Sectimi  2.3  describes  learning,  reuse,  and  the 
utility  fvoblem.  Finally  Secticxi  2.4  presents  four  other  planners  that  reuse  previous  solutions 
and  conq)ares  HINGE  with  them  in  terms  of  their  management  of  die  utility  problem. 
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2.2  Planning  Terms  and  Models 


In  Qiapttf  I,  planning  systems  were  characterized  as  algoridims  that  search  for  a 
sequence  of  operahvs  that  transfcum  a  given  initial  state  into  a  state  in  which  problem  goals 
are  true.  This  section  fomalizes  that  description  and  extends  it  to  eliminate  uxmecessary 
ordering  ccMistraints  on  qierators. 

2.2.1  Representing  Domain  Objects  and  Operators.  Many  problems  encountered 
daily  in  business,  industry,  and  the  military  cm  be  cast  as  planning  problems  and  solved  using 
planning  systems.  To  do  so.  the  problem  must  first  be  m^ped  into  a  formal  representation 
assumed  by  the  planning  system  and  the  output  from  the  planning  system  must  be  interpreted 
in  terms  that  make  sense  to  the  user,  hi  fiiis  research,  it  is  assumed  that  domain  objects  and 
operators  are  represented  by  propositions.  Alfiiough  alternative  representations  exist,  object 
attributes,  relationships  between  objects,  and  changes  in  state  can  most  easily  be  represented 
by  prtqiositkuis.  The  syntax  and  semantics  of  jv^c^xisitums  can  be  found  in  (Gries,  198S). 

Operatcvs  are  identified  widi  names.  It  is  assumed  that  there  is  an  nu^ping  between 
names  of  operates  and  acticxis  so  that  the  output  of  a  planning  system  may  be  interpreted  by 
the  user. 

222  Definitions.  To  specify  a  planning  problem,  definitimis  are  required  fw  the 
fundamental  conc^ts  of  state  and  problem  goal  and  frar  the  meaning  of  change  caused  by 
actum. 

Definition:  State  A  state  is  a  unique  set  of  non-negated  propositions  that 
describes  all  objects  of  intraest  in  a  domain  in  terms  of  pertinrat  relationships 
between  objects  and  attribute  values  for  each  object 

Definition:  Problem  Goal  In  file  ctmtext  of  a  planning  problem,  a  problem 
goal  is  a  propositum  that  indicates  a  desired  value  fOT  a  particular  relationshq) 
between  objects  or  an  attribute  value  for  an  object 
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Because  stales  are  described  by  unique  sets  of  propositions,  each  state  is  distinguishable  from 
all  odier  states.  A  set  of  problem  goals  may  be  included  in  multiple  states;  any  state  that 
includes  all  die  problem  goals  is  referred  to  as  a  “goal  state.” 

An  qperadM'  is  a  data  structure  used  by  die  planner  that  has  die  interpretation  of  an 
acticHi.  An  t^ierator  c^tures  the  important  attributes  of  action:  the  changes  in  state  caused 
by  the  actimi  (effects)  and  the  propositions  that  must  be  true  before  the  action  is  feasible 
(preumdititHis).  Applying  an  operator  has  the  same  in^lications  as  performing  an  actitm. 

Definitioii:  Opmitor  An  operator  consists  of  an  operator  name,  a  set  of 
prectxiditions,  and  a  set  of  effects.  Each  precondition  and  effect  is  a  proposition 
which  may  be  negated  or  non-negated. 

Definitioii:  Applying  an  Operator  An  operator  may  be  applied  in  any  state 
which  includes  all  of  its  preconditions.  When  applied  in  a  State  J,  an  operator 
causes  a  transiticHi  to  a  new  state.  Stated,  which  is  found  by  deleting  all  negated 
operator  effects  from  State  J  and  adding  all  non-negated  t^ierator  effects.  Mo-e 
genaaUy,  an  (^ator  may  be  applied  whenever  its  precmiditions  can  be  shown 
to  hold. 

To  save  storage  space,  practical  planning  systems  take  (^leratcn  schemas,  rather  than 
operators,  as  inputs.  An  operator  schema  includes  variables  in  its  name  and  as  qierands  in 
its  preomditicHis  or  effects.  Fen  example,  ctxisida’  an  operator  which  includes  die  propositiem 
SOME-RELAriONSHIP(A,  B,  C)  in  eidier  its  prectMiditums  or  effects.  Radier  than  require 
opo'ators  that  are  necessary  to  represent  a  domain  of  n  objects  such  as  A,  B,  and  C, 
a  planning  system  diat  takes  qieratcx  schemas  as  inputs  requires  just  one  tqierator  schema 
with  a  predicate  of  the  form  SOME-RELAriONSHIP(?-X,  ?-Y,  ?-2),  where  ?-X.  ?-Y,  and  ?-Z 
are  variables.  Sudi  a  planning  system  finds  operates  by  instantiating  all  variables  in  useful 
(^lo'ator  sdiemas,  unifying  predicates  in  each  qierator  sdiema  with  proposituxis  in  a  state  ot  a 
set  of  goals.  To  guide  die  unification  process,  an  op^ator  schema  must  also  specify  constraints 
(n  die  values  of  variables.  These  cemstraints  are  used  to  keqi  different  variables  from  unifying 
widi  die  same  constant  (variable  separation)  or  to  represent  domain  thetny  required  to  define 
die  operator  schema. 
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Defiiiitk»:  Operator  Schema  An  operator  schema  consists  of  an  operator 
name  which  possibly  contains  variables;  a  set  of  preconditions,  each  of  which 
is  a  negated  or  ntui-negated  predicate;  a  set  of  effects,  each  of  which  is  either 
a  negated  or  non-negated  predicate;  and  a  set  of  constraints  on  the  values  for 
variables.  An  operator  schema  must  cmitain  at  least  one  variable  as  an  operand 
to  a  precondititHi  or  effect 

A  plan  is  often  thought  of  as  a  sequence  of  operators,  but  a  total  ordering  on  all  operators 
is  not  always  required. 

Definition:  Plan  A  plan  is  a  set  of  operators  and  a  set  of  ordering  constraints 
between  op^ators.  where  an  ordering  constraint  is  denoted  by  01  x  02, 
meaning  that  the  operator  01  necessarily  precedes  the  operatm^  02  in  the  plan. 

Note  diat  the  definition  above  does  not  require  a  plan  to  be  able  to  solve  a  problem  or  even  to 
be  executable  from  an  initial  state;  instead,  a  plan  is  file  current  product  of  a  planning  system. 

If  a  planning  algorithm  searches  in  a  space  of  states,  then  it  is  easy  to  determine 
if  an  operator  can  be  applied  by  examining  the  current  state.  However,  as  discussed  in 
Section  2.2.3,  searching  in  a  space  of  states  with  a  sinqile  search  strategy  is  inherently  less 
flexible  than  seardiing  in  a  space  of  plans.  Whra  searching  in  a  space  of  plans,  some  data 
structure  is  required  to  ensure  that  operators  can  be  applied.  For  planners  fiiat  search  in  a 
space  of  plans,  two  inqxutant  concepts  useful  in  fiiis  regard  are  establishment  and  clobberer. 
An  establishment  is  a  data  structure  used  show  that  prectmditions  hold.  A  clobberer  is  an 
(^rator  that  “threatens”  or  possibly  invalidates  an  establishment 

Definition:  Establishment  Wlfii  respect  to  a  particular  plan,  an  establishment 
is  a  trqile  <  E,  p,  C  >  where  p  is  a  proposition,  is  an  operator  that  includes 
p  in  its  effects,  C  is  an  operator  that  includes  p  in  its  preconditions,  and  the  set  of 
ordering  constraints  in  the  plan  includes  £  -<  C. 

Definition:  Clobberer  With  respect  to  a  particular  plan  and  an  establishment 
<  E,  p,  C  >,  a  clobberer  is  an  operator  Clob  whose  effects  include  the  negation 
of  p  and  neifiier  Clob  -<  E  nor  C  -<  Clob  ate  in  the  ordering  constraints  of 
fire  plan. 

ff  an  establishment  <  E,  p,  C  >  exists,  fiien  the  operator  E  is  called  an  establish^’,  while  the 
opo’ator  C  is  referred  to  as  file  consumer.  E  is  said  to  establish  p.  Finding  an  establishment  for 
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a  particular  precondition  is  called  establishing  the  precondition.  Furtiiermore,  if  a  clobberer 
Clob  exists  witii  respect  to  a  plan  and  an  establishment  <  E,  p,  C  >,  then  Cloh  is  said  to 
clobber  die  establishment. 

Chapman’s  modal  truth  criterion  defines  a  method  for  determining  if  a  given  proposition 
is  true  in  a  given  state  or  in  the  context  of  a  particular  plan  (Chapman.  1987:340).  The 
definitions  below  captures  the  essence  of  die  modal  truth  criterion  and  are  useful  for  searching 
in  a  space  of  plans. 

Definition:  Operator  Applicability  An  operator  is  ^plicable  in  a  plan  if  all 
of  its  preconditions  are  true  in  die  state  in  which  the  operator  is  ^plied.  For 
searching  in  a  state  of  plans,  an  operator  C  is  ^plicable  if  every  precondition  of 
C  has  an  establishment  and  there  is  no  clobberer  of  the  establishment  in  the  plan. 


Definition:  Plan  Applicability  A  plan  is  applicable  if  every  op^ator  in  die 
plan  is  ^plicable. 

The  input  to  a  planning  system  is  a  planning  problem  which  includes  an  mitial  state, 
a  set  of  problem  goals,  and  a  set  of  primitive  operators  or  primitive  operator  schemas.  For 
searching  in  a  space  of  plans,  it  is  convenient  to  represent  the  initial  state  and  set  of  problem 
goals  as  an  initial  plan.  A  plan  that  solves  a  planning  problem  is  called  a  solution  and  can  be 
defined  in  terms  of  t^pticability  and  the  initial  plan. 

Definition:  Planning  Problem  A  planning  problem  is  a  3-tuple  <  7,  G,  O  >, 
where  7  is  a  set  of  propositions  that  are  all  true  initially,  G  is  a  set  of  propositions 
diat  must  be  true  in  the  final  state,  and  O  is  a  set  of  operators  or  q)erator  schemas. 

7  is  called  the  initial  state.  Elements  of  G  are  called  problem  goals.  Elements 
of  O  are  called  primitive  operators  or  primitive  operator  schemas  to  distinguish 
them  from  macro  operators  or  macro  operator  schemas  that  are  learned  from  plans 
during  the  course  of  problem  solving. 

Definition:  Initial  Plan  An  initial  plan  of  a  particular  planning  problem  is 
a  plan  that  consists  of  two  operators,  init  and  final,  and  an  ordering  constraint 
init  -<  final;  init  has  no  preconditions,  and  its  effects  are  equal  to  7,  while  final 
has  preconditions  equal  to  G  and  has  no  effects.  All  otho*  operators  in  die  plan 
are  constrained  to  come  after  init  and  before  final. 
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Definition:  Solution  A  plan  is  a  solution  of  a  particular  planning  problem  if  it 
is  ^plicable  and  includes  the  initial  plan  of  the  planning  problem. 


For  a  given  set  of  operators,  there  is  a  one-to-one  correspondence  between  a  planning  problem 
and  its  initial  plan.  A  solution  always  includes  init  and  final,  diese  operators  should  be 
eliminated  when  the  solution  is  interpreted. 

Once  derived,  a  solution  can  be  converted  into  a  macro  operator  as  described  in  Sec¬ 
tions  2.3.1  and  3.8.1. 

Definition:  Macro  operator  A  macro  operator  (or  macro)  is  an  operator  that 
represents  a  solution  to  a  planning  problem  and  is  annotated  widt  the  solution  so 
that  the  maao  operator  may  be  interpreted  by  the  user  of  a  planning  system. 

In  HINGE  and  most  other  practical  planning  systems,  learned  plans  are  stored  as  maao 
opaator  schemas  bodi  to  save  space  and  to  make  learned  plans  useful  for  solving  many  more 
problems.  In  die  context  of  planning  with  reuse,  die  maao  operata  schemas  (or  maaos  that 
arise  from  them)  unda  consideration  by  the  planna  are  called  reuse  candidates  and  represent 
genaalized  solutions  (or  solutions)  to  previously  learned  planning  problems. 

2,23  Search  Space:  States  or  Plans.  Planning  with  simple  search  strategies  in 
a  state-space  has  important  implications  regarding  ordering  constraints  between  opaators 
and  how  operators  are  inserted  into  plans.  Using  a  sinqile  search  mediod  like  depth-first 
search,  every  opaator  in  die  curroit  plan  has  an  ordering  constraint  with  respect  to  every 
otfaa  operator,  and  operators  in  die  plan  are  said  to  be  totally-ordered.  Some  of  these  ordering 
constraints  are  required  to  establish  preconditions  or  eliminate  clobberers,  but  other  ordering 
constraints  are  inqxised  by  die  search  process  itself.  For  example,  consider  the  state  space 
shown  in  Figure  4  and  opaators  defined  by  Figure  S.  Neither  operator  has  any  preconditions. 
Suppose  the  initial  state  is  SJ  (the  enqity  set)  and  die  problem  goals  are  A  and  B  so  that  54  is  a 
goal  state.  Thae  ae  two  different  paths  from  S 1  to  S4.  Howeva,  only  one  path  will  be  found 
if  dqith-first  seach  or  anodia  simple  seach  strategy  is  used.  The  path  found  will  depend 
on  the  planna’s  choice  frxr  die  first  goal  to  achieve,  but  eidia  path  leads  to  a  plan  in  which 
(^laatCHrs  are  totally-ordered.  Either  plan  includes  an  unnecessary  ordering  constraint  between 
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Figure  4.  A  state  space  with  multiple  possible  paths. 
Operator  Oefinitione 


achieve  B 


achieve  A 


preconditions: 

effects 


Figure  S.  The  definitions  of  operators  of  the  state  space  shown  in  Figure  4. 

qierators.  Adding  fills  unnecessary  ordering  constraint  is  a  form  of  over-commitment  by  the 
planning  system. 

Importantly,  searching  in  a  state-space  does  not  inqily  a  total-order  on  operators  for  every 
search  strategy;  it  is  the  combination  of  search  strategy  and  search  sp^  which  determines  this 
pr(^)erty.  For  exan^le,  a  more  sophisticated  search  method  could  be  devised  for  finding  bofii 
paths  shown  in  Figure  4  and  recognizing  that  no  OTdering  ctmstraint  is  required  in  the  plan. 
Generally,  such  a  strategy  requires  look-ahead  and  some  analysis.  For  simple  search  strategies 
that  lack  these  characteristics,  the  choice  of  a  state-space  inqilies  totally-ordered  operators. 
I  call  state-space-searching  planning  systems  that  use  simple  search  strategies  “state-space 
plannos.” 

An  altmiative  to  searching  in  a  state-space  for  a  state  that  includes  problem  goals 
is  to  search  in  a  space  of  plans  until  a  solutitm  is  found.  In  a  space  of  plans,  the  nodes  are 
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plans  ((^)erators  and  ordering  constraints  on  diem)  and  for  refinement  planners^ ,  the  transitions 
between  nodes  are  die  result  of  adding  an  operator  or  adding  an  ordering  constraint  to  establish 
a  precmididon  w  eliminate  a  clobberer.  I  call  such  a  planner  a  “plan-space  planner^.”  Even 
with  single  search  strategies,  plan-space  planners  do  not  inqily  a  total  ordering  on  operators 
and  do  not  add  arbitrary  ordering  constraints  for  the  convenience  of  the  search  strategy. 

It  is  commonly  understood  that  committing  to  planning  decisions  is  inappropriate 
whenever  delaying  die  decision  leads  to  discovering  information  useful  for  making  a  bet¬ 
ter  decision:  this  strategy  is  called  “least  commitment”  Many  researchers  in  the  planning 
community  believe  diat  searching  in  a  space  of  plans  is  more  flexible  and  supports  a  strat¬ 
egy  of  least  commitment  better  than  searching  in  a  space  of  states  (Kambhanqiati,  1992, 
McAllester  and  Rosenblitt,  1991,  Tate,  1977,  Wilkins,  1988). 

223.1  Implications  for  Inserting  Operators  and  Reuse.  The  total  ordering 
of  operators  characteristic  of  state-space  planning  models  requires  that  new  operators  can  only 
be  added  at  one  end  or  die  odier  of  the  sequence  of  operators  in  the  current  plan.  If  the  planner 
searches  forward  from  the  initial  state,  then  op^ators  are  added  to  the  end  of  the  current  plan; 
it  is  also  possible  to  find  one  or  more  goals  states  and  search  backward  to  the  initial  state. 

Because  transitions  in  a  plan-space  only  involve  adding  operators  and  or¬ 
dering  ccmstraints.  new  operates  may  be  inserted  anywhere  in  the  current  plan 
(Kambhanqiati  and  Chen,  1993:515).  For  exanqile.  Figure  6  shows  padis  to  different  op- 


^  Refinement  pbumen  only  add  openuon  or  ordering  constraints,  but  in  general,  transitions  in  a  plan-space 
could  also  result  from  deletion  of  operators  or  ordering  constraints.  In  this  document,  plan-space  planners  are 
assumed  to  be  refinement  planners. 

^Plannmg  models  adikli  search  in  a  space  of  plans  are  often  called  “partial-order”  plarming  models  because 
tiiey  do  not  leriaire  a  total-ordering  on  operators  in  a  plan.  The  name  “partial-order"  distinguishes  such  models 
firmn  odier  models  whidi  insert  arbitrary  ordering  constraints  purely  as  a  convenience  for  the  planning  model. 
However.  Frank  Markham  Brown  pmnts  out  diat  “partial-order”  is  a  estaUished  term  in  logic  with  a  meaning 
incon.sistent  widi  its  usage  in  die  phuming  literature  (Brown,  1990).  hi  logic,  a  partial-order  is  a  reflexive, 
antisymmetric,  transitive  relation;  in  contrast,  die  relation  of  ordering  constraints  on  operators  in  a  plan  is 
irreflexive.  antisymmetric,  and  transitive  (a  quasi-order,  in  logic  parlance).  The  “partial”  part  ai  die  planning 
term  refers  to  die  fiwt  diat  die  relation  of  ordering  constraints  on  operators  is  a  partial  relatioo  (every  operator  is 
not  necessarily  ordered  widi  reflect  to  every  odier  r^erator).  To  avrnd  confusion,  I  call  planners  diat  search  in  a 
QMoe  of  plans  “plan-space  planners.” 
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aaa  aab  ...  aca  ... 

Figure  6.  Allowed  refinements  of  an  operator  sequence  for  operators  a,  b,  and  c 
(Kambhampati  and  Chen.  1993:515). 

erator  sequences.  All  paths  shown  are  possible  refinements  if  a  plan-space  is  searched,  but 
if  a  state-space  is  chosen  instead,  those  paths  fiiat  include  dashed  lines  are  not  possible  wifii 
sinqtle  search  strategies.  For  exploiting  plan  reuse,  Kambhampati  and  Chen  have  recently 
pointed  out  file  potential  usefulness  of  being  able  to  insert  operators  at  any  point  in  file  current 
plan  (as  discussed  in  Section  2.4.3): 

“...we  argue  that  fiie  real  utility  of  using  partial  order  [plan-space]  planning ...  is 
fiiat  it  provides  a  flexible  and  efficient  ability  to  interleave  the  stwed  plans  with 
new  <^)a:ators,  fiiereby  significantly  increasing  the  planner's  ability  to  exploit 
stored  plans.”  (Kambhanqiati  and  Chen,  1993:515) 

By  allowing  maao  insertion  anywhere  in  a  plan,  a  plan-space  planner  allows  reuse  precluded 
by  state-space  plann^. 

2232  Implications  for  Nonlinear  Problems.  The  choice  of  search  space 
can  affect  file  algorithmic  requirements  for  solving  certain  problems  and  the  length  of  file 
resulting  sohitums,  but  searching  in  eifiier  type  of  space  can  be  effective  for  finding  solutitHis. 
evm  ftM*  nonlinear  planning  problems.  Nonlinear  problems  are  those  which  cannot  be  solved 
by  achieving  problem  goals  sequentially,  even  when  goals  are  oxisidered  in  all  possible 
mders.  Ntxilmear  problems  are  caused  by  goal  interactimi:  a  planning  system  adiieves  a 
particular  problem  goal,  but  subsequently  the  goal  is  negated  by  an  operator  used  to  achieve 
anofiier  problem  goal  As  an  exanqile,  the  nonlinear  block-stacking  problem  known  as 
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Sussman’s  Anomaly  is  shown  in  Figure  7.  and  its  state-space  is  shown  in  Figure  8.  In 
Sussman’s  Anomaly,  achieving  either  of  the  problem  goals  first  leads  to  a  state  in  which 
further  progress  depends  on  negating  the  achieved  goal.  For  example,  in  Figure  8.  if  on(A,  B) 
is  achieved  first,  state  X  results  and  on(A.  B)  must  be  negated  to  reach  the  goal  state.  To 
find  a  solution  to  a  nonlinear  problem  like  Sussman’s  Anomaly,  the  planner  must  either 
protect  the  goals  it  achieves  from  being  negated  or  re-achieve  problem  goals  that  have  been 
negated.^  While  plan-space  planners  can  use  either  of  fiiese  methods  with  simple  search 
strategies,  state-space  planners  cannot  protect  goals  without  some  form  of  look-ahead  search 
and  search-space  analysis.  For  exan:q)le.  if  a  sinq>le  state-space  planner  achieves  on(A,  B) 
by  reaching  state  X  in  Figure  8.  file  planner  could  protect  fiiis  goal  by  refusing  to  move 
to  a  state  that  negates  it.  but  doing  so  implies  fiiat  file  planner  will  not  find  file  goal  state. 
The  (Xily  qitkm  for  such  a  state-space  planner  is  to  re-adiieve  goals  that  are  negated.  For 
exan^le.  in  Figure  8.  if  on(A,  B)  is  achieved  first,  the  most  straightforward  path  goes  from 
file  initial  state  to  state  X,  then  to  state  Z.  and  fiien  to  the  goal  state.  The  drawback  with  goal 
re-achievement  is  that  the  resulting  plan  is  longer  than  necessary.  Of  course,  it  is  possible 
to  overcome  fills  drawback  by  writing  a  program  which  analyzes  file  plan  and  shwtens  it,  or 
by  using  a  mme  scqihisticated  search  strategy  using  look-ahead  search  with  analysis  to  avoid 
the  drawback.  Iliese  mefiiods  address  the  fundamoital  disadvantage  of  using  sinqile  search 
methods  in  a  state  space:  over-commitment  to  unnecessary  ordering  constraints  between 
(qia-ators.  Rather  fiian  use  a  complicated  planning  algorithm  to  search  in  a  space  of  states, 
some  planners  including  HINGE  are  based  (m  a  simple  algtaifiim  fiiat  searches  in  a  space  of 
plans  (Kambhampati,  1992,  McAUester  and  Rosenblitt,  1991,  Tate,  1977,  Wilkins,  1988). 

22.4  A  Plan-space  Planning  Model.  McAUester  and  Rosenblitt  have  defined  a 
planning  model  fiiat  searches  in  a  space  of  plans  and  only  adds  operator  ordering  constraints 
that  are  necessary  for  establishing  precondificms  or  eliminating  clobberers.  The  algorifiim 
for  McAUester  and  RosoibUtt’s  planning  model  is  non-deterministic  because  it  leaves  file 

^Some  eaily  plannos  such  as  STRIPS  did  not  use  eidier  of  these  mediods  and  could  not  solve  nonlinear 
protdems  (Rkes  and  Nilsson,  1971). 
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goal  satisfactim  algwithm  unspecified  (McAllester  and  Rosenblitt,  1991).  McAllester  and 
Rosoiblitt’s  algorithm  has  served  as  the  basis  for  several  recent  planners  (Barrett  et  al..  1991. 
Hanks  and  Weld,  1992),  and  HINGE  is  loosely  based  on  this  algorifiim.  A  modified  form  of 
the  algorithm  is  given  by  die  following  procedure  FIND-SOLUTION  whose  arguments  are  a 
plan,  a  set  of  outstanding  goals,  and  a  set  of  establishments.  FIND-SOLUTION  is  defined  as; 

1 .  If  die  current  plan  solves  the  planning  problem,  return  the  current  plan. 

2.  Suppose  C  is  an  q)erati(v  in  die  current  plan,  p  is  a  precondition  of  C,  and  the  set  of 
establishments  includes  <  £,  p,  C  >.  If  diere  is  a  clobberer  Clob  of  <  E,  p,  C  > 
in  the  current  plan,  dien  make  a  new  plan  by  adding  one  of  die  following  ordering 
amstraints.  Recursively  call  FIND-SOLUTION  widi  die  new  plan  and  die  current  set 
of  establishments. 

(a)  Clob  -<  E 

(Jb)  C-<  Clob 

3.  Now  diere  must  be  some  tqieratcM'  (7  in  die  plan  that  has  some  preconditimi  p  widi 
no  associated  establishment  If  possible,  do  mie  of  die  following  and  recursively  call 
FlND-SOLUnON  widi  the  new  plan  and  new  estabtishments  diat  result  Otherwise, 
backtrack. 

(a)  If  the  plan  includes  an  existing  (^ator  El  diat  includes  p  in  its  effects  and 
does  not  include  C  x  £1  in  its  cvdering  constraints,  add  die  ordering  constraint 
El  X  C  to  the  plan.  Add  die  estabtishn^t  <  El,  p,  C  >  to  the  set  of 
establishments. 

(b)  Add  to  the  plan  a  new  c^ieratcx'  E2  diat  includes  p  in  its  effects  and  the  new 
ordering  constraint  E2  x  C.  Add  die  establishment  <  E2,  p,  C  >  to  the  set  of 
est^lishments. 

The  modified  algoridim  above  diffei^  from  McAUesto*  and  Ros^iblitt’s  (xiginal  algo¬ 
rithm  in  that  the  original  algmidim  us^  a  stricter  definitimi  of  “clobbera*”  dian  the  (me 
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in  this  ch^ter  (McAUester  and  Rosenblitt,  1991:636).  This  ch^ter  defines  “clobberer” 
as  Tate  does  (Tate,  1977).  McAUester  and  Rosenblitt  state  their  believe  that  the  mod¬ 
ified  algorifiun  above  wtx^ks  as  weU  or  better  than  their  original  algorithm  in  practice^ 
(McAUester  and  RosenbUtt,  1991:638). 

The  algorithm  above  may  be  altered  for  planning  systems  that  take  t^ator  schemas 
as  inputs  .  rather  than  operafix^.  To  accommodate  (q)erator  schemas,  two  alterations  involv¬ 
ing  unification  are  required.  First,  instead  of  adding  a  new  oparattx  in  step  3(b),  an  q)erator 
schema,  instantiated  so  that  it  includes  p,  is  added.  Sectmd,  inalatercaUtoFIND-SOLUTTON, 
some  of  the  remaining  variables  in  the  op^ ator  schema  may  be  instantiated  either  to  establish 
preconditions  (in  st^  3(a)  or  3(b))  (V  to  eliminate  clobberers  in  the  plan  (in  st^  2).  histantiat- 
ing  a  variable  in  an  (^)erat(X'  schema  is  a  planning  decision,  and,  like  other  planning  decisions, 
it  is  left  unspecified  by  die  algoithm.  (McAUester  and  RosenbUtt,  1991:638) 

22.4.1  Complete  Planning  Systems.  FIND-SOLUTION  is  antm-deterministic 
algorifiim.  To  build  a  planning  system  based  tm  FIND-SOLUTION,  a  search  strategy  must 
be  specified.  If  die  search  strategy  is  systematic  and  exhaustive,  then  the  resulting  planning 
system  wUl  always  return  a  solution  whenever  (xie  exists,  and  the  planning  system  is  said  to 
be  confute.  Gon^leteness  is  clearly  a  ttesirable  charactoistic  firom  the  standpoint  of  a  user 
of  a  planning  system,  but  often  the  exhaustive  search  required  to  make  planners  complete 
also  makes  diem  undesirably  slow,  dkimpletoiess  has  txily  recendy  become  commcmplace  in 
planning  systems,  and  it  is  not  a  property  of  some  plannos  diat  have  been  shown  to  pafrxm 
weU  in  certain  domains  (V^Udns,  1988). 

22.42  Plan  Quality  vs.  Planning  Speed.  This  research  does  not  address  the 
quality  of  plans  (sohitims)  produced  by  a  planning  system.  Plans  are  charactoized  in  many 
ways  including  didr  feasttiiUty,  cost,  flexibility,  and  relative  demand  for  particular  resources. 

^Kambhampati  has  duuacterized  the  difoeaces  between  die  modified  and  unmodified  algorifiun  in  terms 
at  a  trade-off  between  seardi-^iace  redundancy  and  over-cwnmitment  (Kambhampati.  1993).  Kamhhampati’s 
lesnhs  are  consistent  with  McAUester  and  Rosenblitt’s  intuitku. 
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Hie  search  strategy  chosen  for  a  planning  system  greatly  affects  the  plans  returned.  However, 
this  research  does  not  address  plan  charact^tics;  instead,  it  focuses  only  on  the  speed  of 
returning  a  soluti<»,  particularly  in  light  of  die  utility  problem. 

2J  Learning,  Reuse,  and  the  Utility  problem 

Except  undtf  very  restrictive  and  impractical  conditions,  planning  systems  require 
an  amount  of  time  which  varies  exponentially  with  the  number  of  (^leratcx's  in  the  plan 
(Bylander,  1991.  Bylander.  1992).  Theretoe.  building  a  fast  planning  system  depends  on 
finding  a  way  to  make  short  plans,  even  for  conqilex  problems.  The  most  obvious  mediod 
of  making  shwter  plans  is  to  use  more  powerful  (^atms  which  achieve  more  goals.  The 
easiest  mediod  of  finding  such  c^ierators  is  to  learn  them  from  planning  expaience.  as  maCTO 
planners  do. 

23.1  Explanation-based  Learning  (EBL).  Forming  a  macro  qieratm’  schema  from 
a  plan  is  a  form  of  learning  called  explanation-based  learning  (EBL).  There  are  many  otho' 
types  of  learning  that  are  also  axisidered  to  be  EBL  (EUman,  1989).  However,  all  methods 
learn  a  generalized  goal  concept  from  a  single  training  exanqile  using  domain  dierHy  to  guide 
learning.  Ihe  “explanation”  is  a  justificatitxi  that  the  training  exanqile  describes  die  goal 
cmcept  Once  the  generalized  crxicqit  is  learned,  the  conc^t  must  be  rqiresoited  in  such 
a  way  as  to  be  recognizable  to  die  problem  solver,  a  process  called  “operatirHializatiQn.” 
(Dejong  and  Mooney,  198Q 

As  ^plied  to  a  plan,  EBL  results  in  a  macro  operator  schema  diat  r^resents  a  general¬ 
ized  plan.  The  goal  concqit  is  a  plan  for  achieving  some  goals  given  some  initial  state.  The 
training  exanqile  is  a  specific  problem  togedier  widi  its  solution.  The  domain  theory  required 
is  knowledge  about  establishments,  clobberers,  and  plan  ^plicability,  as  well  as  knowledge 
about  domain  objects  and  their  geno-alizatirm.  Knowledge  about  establishments,  clobberers, 
and  plan  tq^licability  fcMms  a  causal  model  fw  explaining  how  die  plan  achieves  problem 
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goals  and  what  prqpositiais  of  the  initial  state  are  required  for  the  plan  to  be  applicable. 
Knowledge  about  domain  objects  helps  to  determine  how  a  plan  should  be  generalized. 

As  an  exanq>le.  suppose  an  EBL  system  is  to  learn  a  plan  for  stacking  blocks.  Consider 
die  block-stacking  training  example  shown  in  Figure  9  and  the  operates  definitions  shown  in 
Figure  10.  Using  this  training  example  and  domain  knowledge,  die  EBL  system  can  prove 
that  die  plan  achieves  problem  goals  and  can  idoitify  blocks  A,  B,  and  C  as  irrelevant  parts  of 
the  initial  state.  Knowledge  of  object  prqierties  allows  the  EBL  system  to  avoid  variablizing 
the  table.  Using  domain  knowledge,  die  EBL  system  can  generalize  die  training  exanqile 
into  a  plan  diat  solves  the  generalized  Sussman’s  Anomaly  as  shown  in  Figure  11.  Finally, 
this  generalized  plan  must  be  c^atumalized  so  diat  it  can  be  recognized  and  reused  by  the 
planner.  The  EBL  system  uses  domain  knowledge  again  to  operationalize  the  generalized 
plan,  resulting  in  the  macro  qperatm  schema  shown  in  Figure  12. 

232  The  Utility  Problem.  The  primary  motivatitni  for  learning  and  reusing  learned 
plans  acquired  from  eiqilanatiai-based  learning  (EBL)  is  to  rahance  planning  speed  over  that 
of  a  generative  planner.  Irwically,  reusing  plans  decreases  planning  speed  in  some  cases,  a 
idiencMnenixi  called  die  utility  problem,  hi  Quqiter  L  the  utility  problem  was  introduced  as 
an  enqiirical  observatiem  cm  planning  systems  diat  reuse  plans.  Actually,  the  utility  problem 
affects  any  system  that  reuses  learned  cmicepts. 

Definition:  Utility  Probleni  F(x  systems  that  seek  to  enhance  performance  by 
reusing  learned  ccmcqits,  die  utility  prcdilem  is  die  possibility  diat  die  system’s 
pecfcxmance  will  degrade  with  additUmal  learning.  For  macro  planners,  die  utility 
problem  is  the  possibility  that  the  planner  will  plan  more  slowly  after  learning 
additional  plans.^ 

Previously,  the  utility  problem  has  beoi  considered  oily  in  terms  of  costs,  rather  than 
in  terms  of  bodi  costs  and  benefits.  No  one  has  developed  a  planning  model  and  algoitfam 
designed  to  overcome  the  utility  prtdilem  by  maeasing  the  likelihood  of  reuse  widiout  future 

'^A  geooadve  jdaoner  learns  no  plans  which  is  certainly  fewer  *!»■"  a  macro  planner.  Ihoefore,  anoUier 
desaqXiaa  of  the  utility  problem  is  diat  a  macro  planner  scnnetiines  performs  worse  than  a  generative  plannw 
which  is  baaed  on  the  same  planning  model 
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Problem  goals:  {on(D.  E),  on(E,  F)} 

Solution:  1.  unstack(F) 

2.  stackfE,  R 

3.  stack(D.  Q 


Figure  9.  A  block-stacking  problem  widi  a  solution. 


unclKk(F) 


dea-^R 
on(F.  P) 


on(F,  tabl«) 

-oo(F.D) 

dear(0) 


stacKE,  F) 


on(E,F) 
•<ori(E,  table) 
-de«r^ 


Figure  10.  OperatCM'  definititMis  corresponding  to  the  block-stacking  problem  sotution  in 
Figure  9. 


Problem  Goals:  {on(?X,  ?Y),  on(?Y.  ?Z)} 
Solution:  1.  unsta^?^ 

3!  sSd3?x!  ?Y) 


Figure  11.  The  generalized  Sussman’s  Anomaly  and  a  generalized  solution. 
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Figure  12.  A  gen^ alized  plan  for  Figure  1 1  op^ationalized  by  representation  as  an  (^jerator 
schema. 

intact  FurthemKH«,  previous  research  has  focused  on  the  cost  of  retrieving  a  good  reuse 
candidate,  rather  than  (m  the  inqthcations  of  insaiing  it  into  a  plan. 

Retrieving  a  reuse  candidate  was  previously  thought  to  require  search,  and  the  speed 
of  retrieval  has  been  thought  to  be  a  nonlinear  function  of  the  size  of  reuse  candidates  or  the 
number  of  them  in  a  library  despite  any  indexing  method  used: 

“It  is  sonKtimes  claimed  that  die  utility  problem  will  be  “solved”  by  the  develop¬ 
ment  of  highly  parallel  hardware  and/or  powerful  indexing  schemes.  This  (pinion 
is  based  mi  the  belief  that  either  of  diese  developmmits  would  make  matching  ex¬ 
tremely  inexpensive.  If  matching  were  inexpensive,  this  would  largely  eliminate 
any  questim  of  EBL’s  utility,  since  EBL,  in  effect,  cmiverts  a  tree  search  problem 
into  a  matdiing  problem.  Howevm,  this  argument  ignores  two  inqiortant  points. 

Hrst,  the  deso^tions  learned  using  EBL  are  neithm  bounded  in  number  nm  size. 
Secmidly,  the  conqiutational  cost  of  matrhing  grows  rapidly  with  the  number  and 
size  of  the  learned  descrqitimis.  Thus,  die  matching  process  requires  search  also 
(sometimes  referred  to  as  knowledge  search).”  (Minton,  1990:369). 

Because  retrieval  has  been  thought  to  require  general  search,  many  researchers  be¬ 
lieve  the  utility  problem  can  mily  be  limited  with  selective  learning  (Greiner,  1989, 
Markovitch  and  Scott,  1989,  Minton,  1985,  Minton,  1990).  hi  selective  learning,  derived 
coQcqits  (plans)  are  evaluated  using  a  heuristic  function  that  predicts  the  value  of  the  concept 
in  future  prtdilem  solving.  Concepts  widiout  sufficient  predicted  future  value  are  not  learned. 
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InqxMtantly.  retrieval  does  not  always  require  search.  Furthermore,  when  search  is 
required,  ntamally  a  specific  method  of  search  can  be  used;  this  specific  method  may  be  a 
polynomial  algorifiun,  rattier  than  an  exptMiential  algorittim  required  for  general  search.  The 
con^iutational  requirements  for  retrieval  depend  on  the  amount  of  infratnatitui  available;  if 
complete  inf<vmati<ni  exists,  ttien  search  is  avoidable  w  if  partial  informatkxi  exists,  ttien 
general  search  may  be  avoided.  Recently.  Vel<^  has  addressed  the  utility  problem  using  an 
efficient  retrieval  method  to  find  reuse  candidates  that  are  likely  to  save  search  in  a  case-based 
planner  (Veloso,  1992).  Veloso’s  ^proach  is  discussed  in  Section  2.4.4. 

While  focusing  on  ttie  costs  of  retrieval,  past  research  has  neglected  ottier  costs  that 
cmitribute  to  the  utility  problem.  These  other  costs  are  associated  with  testing  reuse  candidates 
to  see  if  they  make  planning  faster.  Previously,  no  (me  has  found  a  way  to  guarantee  that 
reusing  a  retrieved  candidate  will  lead  to  a  soluticm.  Furthermore,  there  has  been  no  attempt 
to  quantify  the  amount  of  work  entailed  by  inserting  a  reuse  candidate  into  a  plan.  Instead, 
previous  mettuxls  have  been  able  to  increase  the  probability  that  a  reuse  candidate  wiU  lead 
to  a  solution  and  increase  the  likelih(xxl  that  inserting  a  reuse  candidate  will  cause  little  or  no 
planning  w(xk  in  ttie  future.  A  useful  definiticm  that  characterizes  the  “gotxlness’*  of  a  reuse 
candidate  is: 

Definitioii:  Appropriateness  A  reuse  candidate  is  appropriate  if  the  decision 
to  insert  it  into  a  plan  leads  to  a  problem  solution  and  does  not  have  to  be  retracted 
befcve  a  sohiticm  is  found. 

2.4  Related  Research 

Approaches  to  limit  the  utility  problem  while  planning  with  reuse  have  included 

1.  selective  learning  and  unccmstrained  reuse  of  macro  curators  in  a  state-space  planner; 

2.  reuse  of  cat  past  plan  witti  modification  in  a  case-based  planner; 

3.  unc(mstrained  learning  and  reuse  of  macro  operators  in  a  plan-space  planner;  and 

4.  unctxistrained  learning  and  arbitrarily  constrained  reuse  of  plans  in  a  state-space  planner. 
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N(»ie  of  these  methods  promotes  reuse  while  limiting  the  costs  associated  with  die 
utility  problem  as  well  as  the  mediods  used  in  HINGE.  Selective  learning  can  preclude  reuse 
that  is  possible  if  all  plans  are  learned.  Arbitrarily  limiting  the  number  of  reuse  candidates 
considered,  as  die  second  and  fourdi  methods  above  do.  also  precludes  reuse  in  some  cases 
because  a  good  reuse  candidate  may  be  known,  but  not  be  found.  State-space  planning  inches 
diat  reuse  candidates  cannot  be  inserted  except  at  one  end  of  the  sequence  of  operators  that 
represent  die  current  plan.  Hence,  a  state-space  planner  precludes  reuse  that  depends  on 
interleaving  tqierators.  a  capability  allowed  by  a  plan-space  planner. 

In  contrast  to  these  methods.  HINGE  is  a  plan-space  planner  that  uses  unconstrained 
learning  and  selective  reuse.  Selective  reuse  considers  only  appropriate  reuse  candidates; 
because  die  candidates  are  appropriate,  limiting  the  number  of  reuse  candidates  considered 
never  precludes  reuse.  When  the  available  information  is  insufficient  for  selective  reuse. 
HINGE  uses  a  relaxed  fexm  of  selective  reuse  which  also  uses  appropriateness  as  a  foundation 
for  constraining  the  number  of  reuse  candidates  considered.  Selective  reuse  is  described  in 
Section  4.S.  In  contrast,  the  second  and  fourdi  mediods  above  use  weaker  mediods  to  identify 
good  reuse  candidates. 

2.4.1  MORRIS:  Selective  Macro  Learning.  In  1985.  Minton  studied  reuse  in 
STRIPS,  an  early  state-space  macro  planner  diat  suffered  greatly  from  the  utility  problem 
(Pikes  and  Nilsson.  1971.  Pikes  et  al.,  1972).  Prom  his  analysis.  Minton  decided  that  STRIPS 
was  performing  poorly  because  it  learned  too  many  worthless  plans.  Minton  felt  that  STRIPS 
would  produce  plans  faster  if  only  it  learned  few^  plans,  particularly  diose  with  more  value 
to  future  problem  solving.  To  test  his  idea.  Minton  built  a  STRIPS-based  planner  called 
MORRIS  that  learned  only  two  types  of  macros:  those  that  were  most  commonly  used  in  the 
experience  of  the  planner  and  those  that  represented  “non-obvious”  solutions,  implying  that 
tiiey  saved  a  large  amount  of  search.  Minttm  compared  the  performance  of  MORRIS  with 
that  of  a  geno'ative  planner  and  a  STRIPS-like  planner  that  uses  a  non-selective  EBL  system. 
His  results  showed  die  value  of  selective  learning  for  reducing  the  effect  of  the  utility  problem 
and  improving  reuse-related  perfrxmance.  (Minton.  1985) 
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The  retrieval  methods  used  in  STRIPS  and  MORRIS  search  for  macros  linearly.  Each 
macro  in  the  library  is  a  potential  reuse  candidate  that  must  be  tested  to  see  if  it  achieves 
desired  goals.  In  MORRIS,  the  utility  problem  is  controlled  because  of  two  attributes  of 
selective  learning; 

1.  limiting  the  number  of  macros  learned  keeps  the  library  small  and  reduces  the  time 
required  to  retrieve  a  good  reuse  candidate  and 

2.  learning  non-obvious  solutions  results  in  powerful  macros  which  enhance  the  probability 
that  reuse  will  dramatically  improve  performance. 

Selective  learning  also  contains  die  expansion  of  the  search  space  to  some  extent  because  the 
smaller  library  diat  results  means  fewer  reuse  candidates  are  available  to  consider.  However, 
diis  effect  is  coincidental  and  is  not  the  primary  focus  of  selective  learning.  With  its  policy  of 
selective  reuse,  HINGE  considers  only  a  small  fraction  of  die  macros  in  the  library  and  dius 
limits  die  search-space  expansion  much  better  than  MORRIS  does.  In  contrast  to  selective 
reuse,  selective  learning  neglects  the  inqiact  of  inserting  a  particular  macro  into  a  plan  and 
thus  does  not  limit  the  future  work  die  planner  will  have  to  do. 

The  biggest  disadvantage  of  selective  learning  is  that  some  concept  diat  would  be  useful 
on  a  future  problem  may  not  be  learned.  It  is  difficult  to  accurately  predict  the  needs  of  future 
problems  in  some  domains.  MORRIS  uses  a  heuristic  metric  to  determine  if  a  macro  or  rule 
should  be  learned,  but  the  heuristic  may  fail,  hi  contrast  to  selective  learning,  selective  reuse 
does  not  attempt  to  predict  future  value  of  a  plan;  instead,  it  learns  all  plans  and  filters  reuse 
candidates  when  the  future  “arrives,”  along  with  problem  information  useful  for  die  filtering 
process.  By  learning  indLxriminately,  HINGE  avoids  die  work  of  predicting  future  utility  and 
never  fails  to  learn  a  concept  that  is  useful  on  a  future  problem. 

Like  STRIPS,  MORRIS  is  a  state-space  planner.  As  described  in  Section  2.2.3,  a 
state-space  planner  must  add  operators  to  one  end  of  a  sequence  of  operators  dial  constitutes 
die  current  plan,  while  a  plan-space  plann.  can  insert  operators  at  any  point  in  the  plan. 
Therefore,  because  of  the  order  in  which  it  achieves  goals,  MORRIS  precludes  reuse  in  some 


29 


cases  Aat  HINGE  (a  plan-space  planner)  does  not  Minton  has  applied  selective  learning  to 
another  problem-solver,  Prodigy/EBL,  with  the  same  success  (Minton,  1990).  Prodigy/EBL 
selectively  learns  control  rules  that  can  describe  a  broad  range  of  concepts  and  learns  from 
both  successes  and  failures.  T.ike  MORRIS,  Prodigy/EBL  is  a  state-space  planner  and  is 
inh^ently  less  flexible  in  allowing  reuse  dian  HINGE. 

Primarily  because  they  have  no  special  mechanism  to  do  so,  neither  MORRIS  nor 
PRODiGY/EBLcan  solve  nonlinear  problems.  NoLimit  an  algorithm  that  extends  the  search  al¬ 
gorithm  of  Prodigy/EBL,  can  solve  nonlinear  problems,  but  NoLimit  is  a  state-space  planner, 
and  thus  precludes  reuse  allowed  by  HINGE  (Kambhanq>ati  and  Chen,  1993,  Veloso,  1992). 
HINGE  is  a  plan-^ace  planner  and  can  solve  nonlinear  problems  without  any  special  mecha¬ 
nism. 


242  CHEF:  Heuristic  Case-based  Planning.  Case-based  reastming  is  an  ef¬ 
fective  method  of  improving  perfcxmance  through  reuse  that  became  popular  during  the 
mid-1980s.  In  flie  context  of  case-based  planning,  a  “case”  is  a  plan  or  generalized  plan 
alcHig  wifli  the  problem  it  solves  and  a  rqiresentation  of  the  planning  process  used  to 
produce  die  solution.  Many  researchers  have  devel(^)ed  case-based  reasmiing  systems 
(Kolodner,  1983,  Kolodner  etal.,  1985.  Kottm,  1988),  and  a  wealth  of  research  foundations 
have  been  laid  fm*  memory-based  problem  solving  by  others  ((TarbrnteU,  1983,  Schank,  1977). 
There  are  differences  betweoi  case-based  systems,  but  Hanunond’s  (ZHEF  is  representative 
of  many  case-based  planning  systems. 

In  contrast  to  MORRIS.  CHEF  and  other  case-based  reasoners  do  not  regulate  learning; 
instead,  they  work  by  very  selective  reuse.  Whra  CTIEF  is  assigned  a  new  problem,  features 
of  the  problem  and  a  heuristic  retrieval  procedure  are  used  to  retrieve  die  solutitm  for  a  very 
similar  previmis  problem  encountered.  Once  retrieved,  die  past  problem  is  analyzed  by  a  set 
of  heuristic  critics  to  identify  changes  that  are  required  to  make  the  old  solution  fit  the  current 
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problem.  After  modifications  are  identified,  a  set  of  heuristic  repair  procedures  is  unleashed 
to  perform  them.^  (Hammond.  1990) 

It  is  interesting  to  note  that  the  granularity  of  reuse  candidates  is  also  constrained  in 
case-based  planning.  If  there  is  a  library  case  that  matches  a  large  part  of  the  current  problem, 
then  it  will  be  reused.  Smaller  cases  that  fit  a  smaller  part  of  the  problem  (and  thus  could 
save  a  smaller  amount  of  search)  are  ignored,  even  when  no  larger  case  can  be  found.  Thus, 
case-based  planners  occasionally  ignore  a  beneficial  reuse  opportunity. 

The  utility  problem  arises  in  CHEF  in  plan  retrieval  and  diuing  modification  if  die 
required  plan  modifications  are  ultimately  impossible,  but  duriag  both  retrieval  and  modifica¬ 
tion,  the  effects  can  be  minimized  at  the  risk  of  precluding  reuse.  Learning  a  large  number  of 
plans  impacts  retrieval,  but  with  proper  indexing,  retrieval  performance  degrades  sub-linearly 
witii  tile  growtii  of  the  case  library.  Case-based  planning  is  robust  with  respect  to  the  reuse 
candidate.  It  is  not  necessary  to  get  tiie  solution  to  the  most  similar  previously-encountered 
problem,  although  performance  in^rovement  varies  with  the  modifications  required;  reusing 
a  solution  to  any  similar  past  planning  problem  will  often  result  in  a  performance  inqirove- 
ment  Therefore,  retrieval  in  case-based  planners  can  be  less  specific  and  fast^  than  if  tiie 
best  candidate  were  required.  As  in  STRIPS  and  MORRIS  there  is  no  guarantee  that  a  reuse 
candidate  is  tppropriate.  The  choice  to  reuse  a  case  may  eventually  have  to  be  retracted 
if  tile  required  plan  modifications  cannot  be  made;  if  so,  tiien  the  time  spent  on  case-based 
planning  is  wasted.  There  is  no  limit  on  the  amount  of  work  needed  to  modify  a  retrieved 
plan  in  case-based  planning.  However,  by  retrieving  a  plan  tiiat  worked  on  a  similar  problem, 
case-based  planners  increase  the  likelihood  titat  the  plan  is  appropriate  for  reuse  and  requires  a 
limited  amount  of  modification.  There  are  different  strategies  for  handling  failure  in  diffident 
case-based  plann^.  Often,  aft^  failure  in  the  case-based  component,  the  problem  is  solved 

^Kambhampati’s  Boar  planner  uses  an  analytic  causal  model  consisting  of  tiie  set  of  precooditioa  estab¬ 
lishments  to  letzieve  reuse  candidates  and  identify  changes  to  be  made  by  a  generative  planner.  Unis  Friar 
is  a  case-based  planner  tiiat  effectively  eliminates  dependence  on  heuristic  procedures  typically  used  in  other 
case-based  plannos.  (Kambhampati  and  Hendler.  19%) 
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by  a  generative  con^nent;  this  strategy  contains  the  utility  problem  better  than  re-i^plying 
the  case-based  planner  with  an  alternative  case. 


IVpically.  the  case-based  approach  does  not  allow  reuse  of  multiple  cases.  Thus,  a 
typical  case-based  planner  is  not  useful  for  problems  that  would  benefit  from  reusing  two  or 
more  solutions  from  past  experience.  For  example,  learning  problems  in  CHEF’s  domain  of 
Szechwan  cooking  cannot  help  CHEF  solve  a  problem  fiiat  involves  goals  of  making  diimer 
and  keeping  file  kitchen  clean,  even  when  CHEF  also  learns  plans  for  keeping  kitchens  clean. 
Because  there  are  many  more  problem  confiiinations  than  there  are  problems,  it  is  important 
to  be  able  to  solve  combination-type  problems.  Recently  Veloso  has  developed  a  unique 
case-based  planner  that  allows  reuse  of  multiple  cases  (Veloso.  1992).  (See  Section  2.4.4). 

Unlike  typical  case-based  planners,  HINGE  and  other  macro  planners  can  reuse  multiple 
sohitians  firom  previously  encountered  problems.  HINGE  can  insert  multq)le  macros  because 
it  decomposes  any  set  of  goals  which  cannot  be  achieved  directly.  Case-based  planners  have 
not  used  deconqxKition.  mainly  because  the  largest  benefit  from  reuse  corresponds  to  reusing 
the  largest  candidate.  C^-based  planners  focus  on  the  effort  needed  to  modify  a  known 
solutKHi.  hi  contrast,  HINGE  and  ofiier  macro  planners  avoid  modification  entirely,  histead 
they  look  for  ofii^  (^portunities  for  reuse  on  the  remaining  sub-problems.  Even  though  these 
sub-prt^lems  are  smaller  and  the  potential  benefit  from  reuse  is  diminished,  reuse  is  still  likely 
to  be  cheaper  fiian  generation. 

2.43  SNLP+EBL:  A  Plan-space  Macro  Planner.  At  about  the  same 

time  fiiat  HINGE  was  developed,  Kambhampati  and  Chen  assembled  a  plan-space 
macro  planner  to  study  the  usefulness  of  such  a  planner  for  EBL-based  plan  reuse 
(Kambhanqiati  and  Chen,  1993).  Kambhanqiati  and  Chen’s  macro  planner  was  ctmstructed 
by  adding  an  EBL  system  to  the  SNIP  generafive  planner  which  inqilements  McAllester  and 
Rosenblitt’s  planning  model  (Barrett  et  al.,  1991).  For  the  purposes  of  this  discussion,  fiiis 
planner  will  be  called  “SNLP-i-EBL.” 
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Because  they  are  both  plan-space  macro  planners.  SNLP+EBL  and  HINGE  allow  reuse 
precluded  by  stale-space  planners  and  typical  case-based  planners.  SNLP+EBL  and  HINGE 
are  unique  amtuig  macro  planners  in  dieir  special  ability,  during  planning,  to  expand  macros 
in  a  plan.  To  expand  a  macro,  die  macro  is  replaced  by  the  plan  it  represents  (and  is  annotated 
with).  Expanding  two  macros  allows  interleaving  dieir  primitive  operators,  allowing  ordering 
ccHistraints  and  reuse  that  would  not  be  possible  without  expanding  maaos.  Because  diey  are 
plan-space  planners,  neither  SNLP+EBL  ntu-  HINGE  require  any  special  mechanism  to  do  the 
inttf leaving  (just  as  they  do  not  require  a  special  mechanism  to  solve  nonlinear  problems  or 
to  be  conqilete  planners).  Furthermore,  HINGE  and  SNLP+EBL  can  make  use  of  multqile 
reuse  candidates  of  any  size  to  solve  any  planning  problem,  unlike  case-based  planners  such 
as  CHEF. 

Unlike  HINGE.  SNLP+EBL  was  not  designed  to  ctmtain  the  utility  problem.  HINGE 
uses  an  efhdentretrieval  method  based  on  hashing  and  specific  indices.  In  contrast,  SNLP+EBL 
searches  the  library  linearly  to  find  reuse  candidates  fiiat  match  the  most  goals.^  Retrieval 
based  (xi  goal  matching  is  not  as  selective  as  retrieval  based  on  {q)propriatraess  for  reasmis 
desoibed  in  SectUm  4.2.  Therefore,  SNLP+EBL  does  not  strcmgly  constrain  file  amount  of 
future  work  entailed  by  insertion  of  a  reuse  candidate,  as  selective  reuse  in  of  HINGE  does. 
Finally,  there  is  no  restriction  on  the  number  of  reuse  candidates  considered.  Fm  these  reasons, 
SNLP+EBL  does  not  contain  the  utility  problem  nearly  as  well  as  HINGE  does. 

2.44  Prodigy/Analogy.  A  State-space  Case-based  Planner.  Recently,  Veloso 
has  extoided  PRODIGY  (a  state-space  plann^)  to  reuse  plans  and  control  informatitm  found 
when  deriving  file  plans,  fiius  combining  elements  of  macro  planning  and  case-based  planning. 
This  planner,  Prodigy/Analogy,  extends  case-based  planning  by  allowing  multiple-case 
reuse.  Prodigy/Analogy  also  uses  an  efficient  retrieval  mefiiod  to  find  similar  past  plans. 
Like  other  case-based  planners,  Prodigy/Analogy  addresses  the  utility  problem  by  depend- 

^KarnWuimpati  has  extensively  researched  r^rieval  based  oo  a  stroog  causal  model  which  could  be  used 
to  io^cove  SNLP-tEBL  (Kambhampati.  1990).  However.  Kamhhampati  and  Chen  developed  SNLP4HBL  to 
conqmre  a  plan-space  planner  widi  odier  planners  in  terms  of  dieir  alnlity  to  reuse  plans.  Hus  purpose  was  well 
served  by  fl»esmq>le  storage  and  retrieval  strategy  described  in  (Kamhhampati  and  Chen.  1993:517). 
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ing  its  efficient  retrieval  method  and  its  similarity-based  estimate  of  “goodness"  fOT  the 
retrieval  candidate  (Veloso,  1992). 


Ihe  hash-based  retrieval  mediod  of  Prodigy/Analogy  is  similar  to  the  one  used 
in  HINGE.  Unlike  HINGE,  Prodigy/Analogy  uses  a  discrimination  net  to  avoid  testing 
preconditions  coimnon  to  two  reuse  candidate  twice.  HINGE’S  retrieval  method  is  a  bit  less 
efficient,  but  it  was  simpler  to  iii:q)lement  and  is  sufficient  for  die  purposes  of  this  research. 
Both  methods  require  polynomial  time  (0(n^))  and  vary  with  die  size  of  the  initial  state,  die 
average  number  of  macro  preconditions,  and  the  number  of  macros  hashed  to  the  same  set  of 
goals  achieved  (Veloso,  1992). 

Prodigy/Analogy  uses  its  retrieval  method  to  find  macros  diat  represent  similar  past 
plans.  Therefore,  the  reuse  candidates  Prodigy/Analogy  omsiders  are  not  guaranteed  to 
work  and  do  not  imply  any  limitation  on  the  work  entailed  by  inserting  them  into  a  plan.  In 
contrast,  using  its  policy  of  selective  reuse,  HINGE  retrieves  candidates  guaranteed  to  be  ap¬ 
propriate  fcv  a  problem  or  sub-problem.  For  reasons  discussed  in  Sectitm  4.2,  ^prt^riateness 
also  in^ilies  that  no  future  wo-k  will  be  required  by  an  insoted  reuse  candidate.  Because  of 
selective  reuse,  HINGE  omtains  the  utility  problem  better  than  Prodigy/Analogy. 

To  reuse  plans,  Prodigy/Analogy  seardies  in  the  library  f(»r  a  reuse  candidate  diat 
achieves  as  many  goals  as  possible.  If  no  reuse  candidate  can  achieve  all  outstanding  goals, 
then  tile  problem  is  effectively  decomposed  by  finding  a  candidate  which  achieves  some  goals 
and  searching  again  for  a  candidate  whidi  attiueves  the  remaining  goals.^  This  strategy  works 
well  when  tiie  library  coitains  reuse  candidates  which  achieve  all  or  nearly  all  goals.  Other¬ 
wise,  tile  strategy  must  search  through  a  much  larger  space  to  retrieve  a  reuse  candidate  because 
there  are  relatively  many  unique  subsets  when  a  set  of  goals  is  divided  ^proximately  equally. 
Thus,  Prodigy/Analogy  has  high  retrieval  costs  for  problems  that  must  be  decomposed  into 
sub-problems  of  roughly  equal  size.  Because  of  tins  strategy,  Prodigy/Analogy’s  can  effi¬ 
ciently  solving  only  problems  that  are  similar  to  previously  solved  problems,  a  limitation  that 

^Aa  analogous  decooqwsitioa  method  was  developed  for  HINCHB  (see  Section  3.7.3),  but  HINGE  also  uses 
other  metiiods. 
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is  characteristic  of  case-based  planners.  In  oxitrast.  widi  simple  domain  knowledge,  HINGE 
can  efficiently  decompose  problems  into  same-size  chunks,  resulting  in  a  much  more  directed 
search  fcff  reuse  candidates.  Elecompositum  n^thods  developed  for  HINGE  are  described  in 
Sectitm  3.7. 

Finally,  Prodigy/Analogy  is  a  state-space  planner.  As  previously  argued,  state-space 
planners  preclude  reuse  allowed  by  plan-space  planners  such  as  HINGE. 

25  Summary 

This  chapter  introduced  plan-space  planning  and  conq)ared  it  with  state-space  planning 
in  terms  of  the  reuse  allowed.  A  modified  versimi  of  McAUester  and  Rosenblitt’s  plan- 
space  planning  algraifiim  was  presented.  Their  algmifiun  results  in  a  complete  planno*  when 
exhaustive  search  is  used.  To  speed  planning,  nuxe  powerful  operators  must  be  learned 
so  that  even  conq)lex  problems  can  be  solved  using  sht^  plans.  This  ch^ter  introduced 
explanatitm-based  learning  (EBL)  and  discussed  file  utility  problem  fiiat  arises  whoi  reusing 
EBL  omcepts.  InqxHTtantly,  file  utility  problem  has  been  previously  characterized  in  terms 
of  costs,  and  primarily  in  terms  of  the  cost  of  retrieving  a  good  reuse  candidate.  Many 
researchers  believe  fiiat  selective  learning  is  file  only  way  to  contain  the  utility  problem, 
alfiiough  fite  utility  problem  can  also  be  addressed  by  limiting  reuse  to  mie  candidate,  as 
typical  case-based  planners  do,  cm:  with  effidoit  retrieval,  as  Veloso  has  done.  Finally,  four 
related  planning  systems  were  described  in  terms  of  fiieir  ability  to  reuse  plans  and  fiieir 
managemmt  of  file  utility  problem. 

The  utility  problem  rqiresents  a  fundamental  limitation  on  planning  systems  fiiat  try  to 
improve  efficiency  by  reusing  previous  sohiticms.  Howevo’,  selective  learning  is  not  the  (xily 
way  to  omtain  it  Combating  the  utility  problem  requires  an  architecture  designed  to  increase 
the  probability  of  reuse,  while  limiting  the  costs  of  retrieval  and  testing  reuse  candidates. 
Quqiter  ni  describes  a  planning  system  (HINGE)  fiiat  promotes  reuse  wifii  a  flexible  planning 
model  whidi  rqnesents  desired  features  of  reuse  candidates  and  offo^  multqile  (^pmtunities 
for  rwse  through  search  and  deconqiosition.  Chapter  IV  analyzes  file  costs  of  the  utility 
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problem  and  derives  a  retrieval  mechanism  and  a  reuse  policy  that  combine  to  reduce  these 
costs. 
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in.  HINGE 


3.1  Introduction 

Current  methods  for  addressing  die  utility  problem  include  selective  learning  (in  maao 
planning)  and  limiting  the  reuse  to  a  single  candidate  and/or  efficient  retrieval  (in  case-based 
planning).  These  methods  try  to  solve  the  utility  problem  by  limiting  the  cost  of  retrieval  (mt 
by  limiting  expansitm  of  die  search  space.  In  axitrast,  I  treat  die  utility  problem  both  in  terms 
of  costs  and  braefits  from  reuse.  This  duster  describes  HINGE,  a  plan-space  macro  planner 
inqilemoited  to  expl<»«  diis  eclectic  i^proach.  The  chapter  describes  HINGE  in  general, 
but  it  focuses  <hi  HINGE’S  ability  to  promote  search-saving  reuse.  Chapter  IV  describes  two 
methods  designed  to  strictly  limit  costs  diat  attribute  to  die  utility  problem. 

This  ch^ter  describes  HINGE  in  terms  of  its  planning  model  and  search  control  mech¬ 
anism.  As  an  ovmriew,  die  chapter  begins  by  defining  HlNGE’s  planning  model  and  abstract 
operatcxs  used  in  the  model  An  overview  of  HINGE  is  presorted  in  Section  3.3,  followed  by 
a  descr^tun  of  HINGE’S  basic  search  ctmtrol  mechanism  and  a  sinqile  exan^le  of  planning 
(Sectitxis  3.4  sid  3.S).  HlNGE’s  hierarchical  search  control  is  discussed  and  a  set  of  decom- 
positkm  methods  devek^ied  to  support  it  are  desoibed  in  Sections  3.6  and  3.7.  Section  3.8 
presorts  a  brief  descrqitkm  of  HlNGE’s  EBL  con^xment  Finally,  Section  3.9  reitoates  the 
flexibility  of  reuse  diat  HINGE  supports  by  virtue  of  its  planning  model 

32  HINGE’S  planning  model 

To  promote  seardi-saving  reuse,  HINGE  is  based  oa  a  planning  model  that  searches  in 
a  space  of  plans  and  explicitly  rqiresents  goals  that  should  be  achieved  by  a  reuse  candidate. 
HINGEextends  McAllesta  and  Rosenblitt’s  plan-space  model  by  including  abstract  (^atcH^, 
data  structures  for  reixesenting  subsets  of  fire  outstanding  goals.  A  subset  of  goals,  in  turn, 
rqxesents  a  particular  deconqiositicxi  of  a  problem  and  also  describes  goals  a  reuse  candidate 
must  achieve.  Thus,  an  abstract  cqierattx  represents  a  sub-problem  and  can  be  used  to 
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index  a  plan  that  solves  the  sub-problem.  Abstract  operators  may  also  be  used  to  establish 
precmiditions  and  insert  partial  domain  knowledge,  when  it  is  available. 

In  additkm,  HINGE’S  planning  model  was  designed  to  concisely  represent  plans  and 
the  process  of  planning  so  that  the  planner’s  decisitms  can  be  e]q>lained.  F(x  representatimi. 
HINGE  uses  a  single  data  structure  that  holds  the  amtext  of  the  planning  process.  This 
data  structure  includes  the  set  of  establishments  and  (^)erat(M'  cxdering  constraints  from  which 
a  strong  causal  noodel  may  be  derived.  The  planning  process  is  captured  by  a  plan  tree 
diat  records  die  history  of  decisicm  making  and  represents  untried  choices  that  are  useful  if 
backtracking  is  required. 

32.1  A  Data  Structure  for  HINGE’s  planning  model.  For  simplicity,  HINGE’S 
planning  model  uses  (Hily  (me  data  struchire  to  rqiresent  outstanding  goals,  precondition 
establishments,  and  die  current  plan.  This  data  structure,  called  a  validated,  hierarchical 
plan  (VHPLAN,  pronounced  ’v-h-plan’)  also  represents  a  plan  tree  that  is  used  to  make 
decisicms  about  search  ccxitrol  and  c^tures  planning  decisions  which  led  to  the  current  plan.^ 
This  VHPLAN  uses  typed  operatcHS  to  differentiate  actions  and  outstanding  goals.  Abstract 
(^leratc^,  defined  in  die  next  section,  identify  goals  to  achieve,  while  (xmcrete  opo^ators 
doiote  instantiated  operator  schemas  inserted  into  the  plan  to  achieve  goals.  Figure  13  shows 
a  grtqihical  representation  of  a  VHPLAN.  In  the  notation,  precondition  establishments  and 
ordering  constraints  on  operators  are  represented  as  directed  links,  parent/child  relatumships 
betwera  (qierators  are  shown  using  undirected  links,  and  (^ratcnrs  are  represoited  as  boxes 
containing  the  cqierator  name,  preconditions,  and  effects.  A  distinguished  (^ator  labeled 
root  is  die  root  of  the  plan  tree.  The  initial  plan  operators,  init  and  final,  are  conaete  (^rators 
whidi  are  not  part  of  die  plan  tree.  The  leaves  of  die  plan  tree,  along  with  init  and  final, 
represoit  die  currrait  plan.  The  parts  of  the  VHPLAN  (including  establishments)  diat  refer 
to  leaves  of  the  plan  tree,  init,  a-  final  define  ncxies  in  die  space  that  HINGE  searches.  The 

^Hie  tom  •‘validation”  is  anotiier  tenn  for  “establishiiient"  (Kambfaampati  and  Hendler,  1992:206). 
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VHPLAN  is  useful  because  it  compactly  represents  a  great  deal  of  infmination  pertinent  to 
planning. 


Figure  13.  An  exanq)le  of  the  graphical  notation  used  to  denote  a  validated,  hierarchical 
plan  (VHPLAN)  of  HINGE’S  planning  model 


322  Defining  Abstract  Operators  to  Record  Goals  and  Insert  Domain  Knowledge. 
To  record  planning  goals  and  provide  an  insertion  point  for  domain  knowledge,  I  developed 
abstract  operatm^  and  define  them  hwe.  hi  CH'der  to  systematically  search  its  space,  a  planner 
must  somehow  keq)  track  of  file  set  of  outstanding  goals  to  be  achieved.  HINGE’s  model  uses 
abstract  operators  fw  fills  purpose.  Other  operatcK^  are  called  concrete  operates^  and  denote 
actions  fiiat  adiieve  goals; 

Definitkni:  Abstract  and  Concrete  Operatws  An  abstract  operator  is  a 
operator  whose  preconditions  and  effects  are  not  completely  specified  in  file  sense 
that  propositions  are  missing.  If  both  preconditicms  and  effects  are  con^iletely 
specified  for  a  particular  apexatsx,  the  operatcn’  is  called  a  cmicrete  operator. 

Abstract  qpe'ators  support  file  insertion  of  useful  domain  knowledge  when  it  exists.  Mfiiout 
domain  information,  HINGE  abstract  cqierators  never  have  preconditions  and  only  have  those 
effects  fiiat  represoit  planning  goals.  Operator  effects  that  achieve  desired  goals  are  called 
“primary  effects’’  while  those  that  do  not  are  called  “side  effects.’’  If  partial  (or  total)  domain 
knowledge  exists,  fiien  preconditkms  and  side  effects  in  abstract  operators  support  timely 
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insertim  of  die  knowledge.  F(v  example,  suppose  a  goal  G  exists,  but  available  domain 
informatUHi  specifies  that  G  cannot  be  achieved  without  first  thieving  at  least  PI  and  that 
at  least  die  side  effect  SJ  always  accompanies  G.  The  planner  can  use  this  informaticMi  to 
immediately  try  to  achieve  PI  and  ensure  that  SI  does  not  result  in  clobbering.  If  this  domain 
inftvmation  w^e  not  inserted  until  after  the  goal  G  were  identified,  then  these  planning 
actions  would  not  be  possible,  and  there  wmild  be  no  possibility  to  improve  performance  by 
re-directing  search. 

323  Using  Abstract  Operators  in  Establishments.  Representing  goals  widi  an 
abstract  operates  leads  to  the  possibility  diat  an  abstract  operator  may  be  used  to  establish 
new  (qierator  precrxiditions  that  it  was  not  specifically  inserted  to  establish.  On  the  surface, 
diis  tqipears  to  be  overcommitment  (Hi  the  part  of  the  planner  because  there  is  no  guarantee 
that  the  outstanding  goals  represented  by  the  abstract  (^ator  will  be  achieved.  However,  die 
alternative  choices  may  result  in  worse  overcommitment  and  fail  to  take  advantage  of  available 
inf(Hmati(xi.  To  see  this,  ccHisider  the  case  in  which  an  abstract  (^ator  A1  is  die  (Hily  existing 
establisher  for  a  new  c^aUH’s  precondition  p.  Assume  diat  to  avoid  overcommitment,  Al 
should  not  be  used  as  an  establisher.  Two  alternatives  exist  The  first  alternative  is  to  insert  a 
new  abstract  (qierator  A2  which  has  p  as  an  effect,  thus  committing  to  doing  additicmal  work. 
Now  the  planner  must  find  reductions  fcH  both  Al  and  A2.  The  planner  has  also  committed 
to  a  course  of  actUxi  that  assumes  the  failure  of  Al  and  die  accompanying  failure  of  the 
(XXKrete  (^atcH  which  requires  Al.  This  planning  choice  is  based  on  no  new  information 
and  ign(H:es  existing  information  about  goals  to  which  the  planner  is  already  committed.  The 
sec(Hid  alternative  is  to  delay  die  choice  of  establish^’  for  p  until  more  information  becomes 
available.  For  exanqile,  rather  than  establishing  p  widi  Al.  the  planner  could  choose  to 
find  a  reduction  for  Al  that  includes  a  (xmerete  establisher  for  p.  If  a  such  a  reducticHi 
is  found,  then  this  alternative  was  a  p(XH  choice — it  would  have  been  better  to  establish  p 
with  Al.  If  no  such  reduction  is  found,  then  the  planno-  knows  diat  Al  would  have  been 
a  po(H  choice  fcH’  establidiing  p,  but  it  has  no  new  infcnmatitHi  he4>ful  to  finding  a  better 
choice.  The  secemd  alternative  also  ignenes  die  planner’s  previous  commitment  to  goals  and 
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overcoimnits  die  planner  to  searching  for  reducticHis  of  A1 .  When  knowledge  of  goals  exists, 
the  planner  should  make  decisions  based  mi  die  assumption  that  the  goals  will  be  achieved. 
The  two  alternatives  to  diis  approach  disregard  infmmation  and  result  in  overcommitment 
diat  may  be  worse  than  committing  to  using  an  existing  abstract  (^lerator  as  an  establisher. 
Sectims  3.4.1  and  3.4.3  discuss  search  control  in  HINGE  designed  to  limit  the  in^iact  of 
possible  overcommitment  associated  with  abstract  operator  establishment 

32.4  Definitions  Required  by  Abstract  Operators.  Abstract  q;)erators  tyipear  in 
plans,  but  they  should  not  appear  in  a  solutimi  because  they  represmit  goals  and  do  not 
specify  action.  Therefore,  introducing  abstract  operators  requires  a  change  to  the  definition 
of  “solution”  to  avoid  solutions  that  contain  abstract  operators.  More  generally,  abstract 
qpm-atCH^  in  HINGE’S  planning  model  leads  to  a  definition  of  typed  plans. 

Definition:  Abstract  and  Concrete  Hans  An  abstract  plan  is  a  plan  that 
includes  at  least  erne  abstract  operatm.  In  contrast,  a  cmicrete  plan  is  a  plan  that 
includes  only  cmicrete  operators. 

Definition:  Solutkm  A  solution  to  a  particular  planning  problem  is  a  concrete 
plan  which  is  {q)plicable  and  includes  the  initial  plan  of  the  planning  problent 

Finally,  HINGE’S  planning  model  requires  some  method  of  eliminating  abstract  opera- 
tcHS  in  a  plan.  An  abstract  operator  represents  wmk  to  be  done,  and  it  is  inqxutant  to  recognize 
that  it  may  not  be  desirable  to  do  all  the  wmrk  at  once.  The  principle  of  least  commitment 
suggests  that  some  of  the  wmk  could  be  done  mme  efficiently  if  it  is  put  off  until  more  infor¬ 
mation  is  available.  Therefore,  tiie  method  of  abstract  operattM'  r^lacem^t  requires  a  data 
structure  which  is  essentially  a  sub-plan,  possibly  ermtaining  otii^,  new  abstract  operators  to 
represent  die  delayed  WOTk.  Such  a  sub-plan  is  called  a  reduction  in  die  general  case,  a 
decomposition  if  it  is  a  sub-plan  that  contains  (Hily  abstract  qia-ators.  hi  HINGE,  planning  is 
the  process  of  rqilacing  abstract  opa’ators  with  reductions,  or  “reducing”  abstract  operators. 

Definition:  Reduction;  Decomposition  A  reduction  of  an  abstract  operator 
in  a  plan  is  a  set  of  operatCH's  called  reductiem  operators  and  a  set  of  (ordering 
ctmstraints  such  diat  if  the  reduction  rqilaced  the  abstract  operator  in  the  plan 
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1.  the  reduction  operators  would  establish  preconditions  currently  established 
by  the  parent  abstract  operator. 

2.  all  preconditions  of  all  reduction  operators  would  be  established,  and 

3.  no  clobberers  of  any  establishment  would  exist. 

A  decon^)osition  is  a  reduction  that  includes  only  abstract  operators. 

Prom  dlls  definition,  it  should  be  clear  that  reducing  an  abstract  operator  maintains  the 
applicability  of  the  plan. 

The  HINGE  planning  model  represents  planning  as  a  tree  search;  in  the  tree,  a  par¬ 
ent/child  relationship  exists  between  an  abstract  operator  and  its  reduction.  For  example,  in 
Figure  13.  abopl  and  abop2  with  no  ordering  constraints  are  the  reduction,  or  in  fius  case,  the 
decomposition  of  root. 

3.25  The  Non-deterministic  Algorithm  of  HINGE' s  planning  model.  HINGE’S 
planning  algorifiim  is  specified  below  as  die  non-deterministic  algorithm  FIND-PLAN  whose 
argument  is  a  VHPLAN. 

1.  If  the  current  plan  of  the  VHPLAN  is  a  solution  to  the  planning  problem,  return  the 
current  plan. 

2.  Otherwise,  diere  must  be  at  least  cme  abstract  operator  in  the  current  plan.  Select  an 
abstract  opo'ator  A  from  the  current  plan  and  for  A  find  all  reductions.  The  algorithm 
for  finding  reductions  is  McAllester  and  Rosenblitt’s  planning  algorithm  described  in 
Chtqpt^  n.  If  diere  are  more  reductions,  choose  one,  substitute  it  for  die  parent  abstract 
(^lerator  in  the  current  plan,  and  recursively  call  FIND-PLAN  with  this  new  VHPLAN 
that  results;  otherwise,  backtrack. 

Some  differences  between  HINGE’s  algorithm  and  the  modified  versitm  of  McAllester 
and  Rosenblitt’s  algoridim  (Section  2.2.4)  seem  ^parent,  but  they  are  actually  quite  similar. 
The  majcH*  difference  between  die  two  algoridims  is  diat  they  use  different  data  structures 
which  support  diffo-ent  types  of  search  control.  HINGE’s  algoridim  uses  the  VHPLAN  data 
structure  which  includes  a  hierarchical  plan  tree  with  abstract  operators  representing  subsets 
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of  the  outstanding  goals.  The  hierarchical  plan  tree  allows  representation  of  a  particular  set  of 
the  goals  of  equal  depth,  while  abstract  operators  provide  a  representation  for  a  set  of  goals 
that  are  to  be  solved  simultaneously.  Both  of  these  sets  of  goals  are  distinguished  subsets  of 
the  whole  set  of  outstanding  goals  being  considered  by  the  planner.  The  ability  to  represent 
these  two  subsets  of  die  outstanding  goals  is  important  for  directing  search.  Most  importantly, 
representing  a  subset  of  goals  to  be  solved  simultaneously  is  critical  for  representing  die  part  of 
the  current  problem  that  should  be  solved  with  reuse.  In  contrast,  McAllester  and  Rosenblitt’s 
algoridun  uses  the  whole  set  of  outstanding  goals.  This  set  representation  cannot  represent 
a  distinguished  subset  of  goals  and  thus  cannot  indicate  a  subset  of  goals  to  be  achieved 
simultaneously.  With  McAllester  and  Rosenblitt’s  model,  it  is  still  possibly  to  find  a  reuse 
candidate  that  solves  part  of  the  problem,  but  the  data  structures  used  caimot  represent  what 
part  should  be  solved.  The  result  is  that,  if  such  information  exists,  it  cannot  be  used  to  direct 
search.  Section  3.6  discusses  diis  point  further. 

Like  McAllester  and  Rosenblitt’s  algorithm,  the  algorithm  above  is  non-deterministic; 
both  algorithms  use  depdi-first  search,  but  neither  algorithm  specifies  how  to  make  choices 
at  branch  points.  The  iiiq)lementation  of  HINGE  as  a  plaimer  requires  that  search  control 
be  specified,  dius  HINGE’S  planning  model  is  more  general  than  the  HINGE  planner  itself. 
hinge’s  search  control  methods  are  discussed  in  Sections  3.4  and  3.7. 

3J  HINGE  Overview 

HINGE  is  a  plan-space  macro  planner  that  includes  an  explanation-based  learner  for 
learning  new  plans  and  sub-plans.  Because  HINGE  is  a  plan-space  plaimer,  it  can  insert 
operators  anywhere  in  the  plan  and  includes  only  required  ordering  constraints  on  operators  as 
tile  plan  is  developed.  Moreover,  HINGE  can  optionally  eiqiand  a  macro  in  apian,  substituting 
its  OOTesponding  primitive  operators  and  necessary  ordering  constraints  for  the  macro  itself. 
Thus,  HINGE  is  capable  of  interleaving  the  primitive  operators  of  macros  to  solve  planning 
problems.  HINGE  and  SNLP+EBL  are  the  first  planning  systems  that  learn  and  can  reuse 
macros  in  tiiis  way. 
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HINGE’S  search  method  is  hierarchically-structured  on  the  number  of  goals.  By  trying  to 
solve  goals  simultaneously.  HINGE  promotes  reuse  of  more  powerful  macros  before  weaker 
ones.  The  hierarchical  structure  imposed  search  also  facilitates  sub-plan  learning.  If 
goals  caimot  be  solved  simultaneously.  HINGE  incrementally  decon^x)ses  them  using  various 
strategies  which  are  described  in  Section  3.7.  Eventually,  decomposition  results  in  single 
goals  that  HINGE  solves  using  generative  planning.  Because  of  its  planning  model.  HINGE 
is  sound^  and  causal  structure  systematic.^  Because  HINGE  considers  all  reductions  using  an 
exhaustive  search  strategy.  HINGE  is  also  conq)lete. 

HINGE  uses  efficient  storage,  indexing,  and  retrieval  mechanisms  that,  when  the  the 
initial  state  and  problem  goals  are  known,  support  polynomial-time  access  of  macros  that  are 
guaranteed  to  be  ^propriate.  If  the  problem  must  be  decomposed,  then  die  initial  state  and 
problem  goals  are  known  only  for  a  sub-problem;  in  this  case,  the  retrieval  mechanism  allows 
polynomial-time  access  of  macros  likely  to  be  ^propriate.  The  retrieval  mechanism  does  not 
require  general  search  and  thus  does  not  vary  exponentially  with  the  size  of  the  macros  or  die 
number  of  them  in  the  library.  Storage  and  retrieval  in  HINGE  are  described  in  Chapter  IV. 
HINGE  includes  an  EBL  conqwnent  for  learning  macro  operator  schemas  from  the  solutions 
to  planning  problems.  HINGE’s  overall  architecture  is  represented  as  shown  in  Figure  2  on 
page  2.  where  the  output  of  the  learner  is  macro  operator  schema,  radier  than  operators. 

HINGE  is  a  domain-independent  planner  designed  to  use  domain  knowledge  when  it 
exists  to  enhance  performance.  For  exanq7le.  HINGE  allows  modular  insertion  of  domain- 
specific  procedures  for  choosing  an  abstract  operator  for  reduction  (goal  ordering)  and  for 
selecting  a  reduction.  These  procedures  are  not  described  in  this  document  HINGE  also 
performs  better  when  there  is  a  domain-definition  of  object  independence.  For  exan^)le. 
knowledge  of  object  independence  was  us^  to  build  a  more  specific  index  for  the  block¬ 
stacking  problems  which  were  used  to  develop  and  test  HINGE.  Independence  is  also  the  basis 
for  two  decomposition  methods  described  in  Section  3.7.  Although  a  domain-definition  of 

^Soundness  means  diat  any  plan  retuined  is  a  soiutkn  of  the  problem  beina  considered. 

^Each  node  in  die  space  being  searched  is  defined  by  a  plan  and  die  set  of  establishments  associated  with  die 
plan. 
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object  independence  is  helpful  for  improving  planning  performance,  lack  of  domain  knowledge 
does  not  affect  HINGE’s  completeness.  Alternative  methods  of  indexing  and  decomposition 
are  available.  However,  for  learning,  a  definition  of  independence  is  required^,  as  described  in 
Sectimi  2.3.1.  Pot  example,  two  stacks  of  blocks  that  are  botii  on  the  table  are  not  considered 
to  be  related  because  of  the  special  nature  of  the  table.  This  domain  information  is  required  to 
avoid  learning  incorrect  concepts. 

HINGE  is  implemented  in  Common  Lisp  and  CLOS  and  includes  a  number  of  utilities 
for  displaying  and  analyzing  die  planiting  process.  As  a  planning  system  designed  to  support 
further  research.  HINGE  can  easily  be  configured  to  run  in  many  different  modes.  More  than 
twenty  global  variables  are  used  to  vary  HINGE’s  operation;  many  of  the  more  important  of 
these  variables  can  be  set  using  a  sinq)le  graphical  user’s  interface  that  is  part  of  HINGE. 
HINGE  has  been  used  by  three  researchers  to  date.  HINGE  prototype  ^plications  include 
a  scheduling  algtHitiun  for  setting  up  a  meeting  without  violating  constraints  inqx)sed  by  the 
attendees,  a  labor-crew  scheduler  for  a  production  shift,  and  an  air  campaign  planning  system. 

3.4  Search  Control 

HINGE  spends  mudi  of  its  time  doing  bookkeeping  to  maintain  die  plan  tree  and  causal 
model  used  to  ensure  plan  ^plicability  when  reductions  replace  abstract  operators.  These 
processes  are  interesting  to  define  and  chailoiging  to  inclement,  but  they  are  well  under¬ 
stood  (Chtq}man.  1987,  Currie  and  Tate,  1991,  McAUester  and  Rosenblitt,  1991,  Tate,  1977, 
Yang  and  Tenenberg,  1990).  However,  the  search  control  used  in  HINGE  is  unique  and  inter¬ 
esting.  As  stated  in  Section  3.2.S.  the  overall  search  strategy  is  depth-first,  and  chronological 
backtracking  is  used  to  retract  poor  choices.^  Within  this  framework,  there  are  three  places 
which  obviously  inq)act  search  in  the  HINGE  algorithm:  selection  of  the  next  abstract  opera¬ 
tor  for  reduction,  generation  of  reductions,  and  preference-ordering  of  reductions.  By  setting 

is  possible  to  derive  independence  by  analyzing  die  plan  before  learning  as  done  in  Prodigy/Analogy 
(Vekiso.  1992),  but  HINGS  lades  (he  capability  to  do  diis  and  dqiends  on  a  domain-definition  of  iatittpaitkncr 
^HINGE  seardies  its  plan-space  d^th-fiist.  but  as  discus^  in  Section  3.4.1.  HINGE  selects  abstract 
operators  from  die  plan  tree  using  a  breaddi-first  criteiimi.  There  should  be  no  ctniiisiai  about  these  terms 
because  HINGE’s  plan-qiace  is  not  the  plan  tree,  but  rather  die  changes  in  the  leaves  of  the  plan  tree. 
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global  variables,  HINGE  may  be  re-configured  to  alter  its  method  of  doing  each  of  these. 
However.  HINGE’S  default  behavior  is  described  here.  This  default  behavior  is  designed  to: 


•  focus  attention  (HI  high-level  goals 

•  reuse  file  most  powerful  reuse  candidates  first 

•  provide  multiple  opportunities  for  reuse 

•  limit  file  impact  of  future  work 

•  make  the  best  local  decisions  possible. 

3.4.1  Selection  of  an  Abstract  Operator  to  Reduce.  HINGE  selects  abstract 
operators  to  focus  attention  on  important  goals  and  to  promote  reuse  of  the  most  powerful 
maaos  first  Selecting  an  abstract  operator  to  reduce  amounts  to  choosing  the  order  of  goal 
achievement  HINGE’S  default  method  is  to  select  abstract  operators  based  on  fiieir  depth  in 
file  plan  tree:  fiiose  of  least  dqith.  This  strategy  focuses  attention  on  problem  goals  before 
operator  sub-goals  and  avoids  poor  planning  decisions  which  lead  to  blind  alleys  or  locking. 
For  abstract  operattHs  fiiat  are  equidistant  from  the  r(X)t  HINGE  prefers  the  one  with  the 
largest  number  of  effects  because  it  corresponds  to  a  potential  opportunity  to  reuse  the  most 
powerful  reuse  candidate. 

Selecting  abstract  curators  based  on  least  depth  in  fiie  plan  tree  also  reduces  the 
potential  overcommitment  of  depending  on  an  existing  abstract  operator  for  preccmdition 
establishment  Existing  abstract  operators  are  always  higher  in  the  plan  tree  than  new  abstract 
(qierators.  Therefore,  if  an  existing  abstract  operator  is  used  to  establish  a  precondition  of  an 
(^rat(H  external  to  the  abstract  operator’s  reductitm,  testing  fiiis  choice  is  not  delayed  as  it 
might  be  if  abstract  operat(Hs  were  selected  some  other  way. 

3.4.2  Generation  of  Reductions.  In  HINGE,  reductions  take  on  one  of  fiiree  forms 
as  shown  in  Figure  14.  Each  fcnm  is  described  by  file  number  and  types  of  op^ators  in  the 
reduction.  If  the  reduction  includes  a  concrete  op^ator,  then  it  will  be  composed  of  either 
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a  single  conaete  operator  or  a  oHicrete  operator  and  an  abstract  operator.  Otherwise,  the 
reduction  will  be  a  deconqx)sition  and  will  include  multiple  abstract  operators  (often  two  of 
them).  These  choices  represent  die  smallest  ways  to  construct  a  reductitm  and  they  correspmid 
to  three  possibilities  when  trying  to  achieve  goals  represented  by  an  abstract  operatCM’  ’s  effects. 
By  definition,  a  reductirm  is  required  to  achieve  the  effects  of  its  parent  abstract  operator.  If, 
by  instantiating  a  library  operatm'  schema,  a  concrete  operator  can  be  found  that  achieves 
die  effects  of  the  parent  abstract  (^lerator,  dien  this  concrete  cqierator  forms  the  basis  of 
a  non-deconqiosituMi-type  reduction.  If  all  preconditions  of  the  concrete  (^ator  can  be 
established  by  existing  plan  operators,  then  no  oth^  operator  is  required  and  the  reduction 
is  of  the  first  form.  Alternatively,  if  one  or  more  preconditions  of  die  concrete  operator 
cannot  be  established  by  an  existing  plan  c^ator,  dien  HINGE  adds  a  new  abstract  operator 
to  establish  these  preconditions,  and  the  reduction  has  the  second  form.  Reductions  of  the 
second  form  always  have  at  least  one  ordering  constraint  corresponding  to  die  establishments 
made  on  bdialf  of  the  cmicrete  operator.  The  third  form  of  reduction  arises  in  the  event  diat 
the  parrat  abstract  op^atc^  has  multqile  effects  and  no  operator  schema  in  the  library  can 
be  instantiated  to  achieve  all  of  these  effects.  In  this  event,  HINGE  enqiloys  deconqxisition 
methods  described  in  Section  3.7  to  separate  die  goals  represented  by  the  parent’s  effects.  This 
results  in  a  reduction  (a  decomposition)  that  includes  only  abstract  operators  with  no  ordering 
cmistraints  between  any  of  them. 
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To  limit  die  amount  of  future  work,  reductions  are  required  to  achieve  all  effects  of  the 
parent  abstract  operator.  If  a  reduction  fails  to  achieve  all  these  effects,  dien  some  wm'k  is  left 
undtme  and  it  is  impossible  to  quantify  the  additional  wm'k  required.  In  addition  to  limiting 
future  wtvk.  insisting  diat  the  concrete  operaUM^  of  a  reduction  achieve  all  of  die  effects  of  die 
parent  abstract  (^ator  is  consistent  with  a  strategy  of  selective  reuse  and  greatly  suI^)lifies 
die  macro  retrieval  method  used  in  HINGE.  This  method  results  in  far  fewer  reductions  than 
would  odierwise  be  generated.^  If  macros  must  achieve  all  effects  of  the  parent,  then  the 
effects  of  operator  schemas  are  a  useful  index  for  macro  operator  schema  retrieval.  Selective 
reuse  and  maao  retrieval  are  described  in  Cht^ter  IV. 

In  general,  diere  are  many  ways  to  decompose  an  abstract  operator,  and  the  choice 
of  decompositicHi  clearly  affects  the  search  process.  Decomposing  an  abstract  operator  is 
equivalent  to  partitioning  a  set  of  goals.  One  way  to  partition  a  set  of  goals  is  to  make 
each  element  a  singleton  subset  This  approach  corresponds  to  completely  decomposing  an 
abstract  with  n  effects  into  n  abstract  operators  with  1  effect  To  provide  multqile 

(^portunities  fw  reuse,  a  less  drastic  approach  is  required  because  reuse  is  not  possible  unless 
multiple  goals  are  to  be  achieved.  Section  3.7  desoibes  several  deconpositum  methods 
develcped  fm'  HINGE.  These  methods  may  be  used  to  incrementally  decompose  problems 
and  provide  multple  opptHlunities  for  reuse,  witii  the  most  powerful  reuse  attempted  first 

By  definitkm,  no  (perator  of  areduction  may  be  a  clobberer  of  any  existing  establishment 
in  the  plan.  Therefne,  the  process  of  finding  reductions  includes  a  procedure  that  eliminates 
the  possibility  of  clobberers  either  by  adding  ord^g  constraints  or  by  discarding  the  reduction 
(when  adding  ordoing  ctxistraints  is  not  p(»sible).  As  described  in  the  definitum  of  FIND- 
PLAN  (page  42),  reductions  are  found  using  McAllester  and  Rosenblitt’s  algwithm  FIND- 
SOLUTION  (page  19).  Fw  conpleteness,  bofli  alternatives  in  step  2  of  FIND-SOLUTION 
must  be  omsida^ed. 

^Requiring  itductioos  to  achieve  all  efiects  of  Oe  parent  abstract  operator  has  die  disadvantage  diat  some 
operator  sdtemas  that  oonld  be  instantiated  to  achieve  many  of  die  parent’s  effects  mi^  be  overlooked. 
Section  3.7.3  discusses  a  decranposition  method  that  increases  the  visddlity  of  such  operator  sdiemas  and 
promotes  dieir  use. 
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3.42.1  Serendipity  and  Phantom  Operators.  Sometunes  decomposition 
results  in  an  abstract  q)erator  whose  effects,  by  coincidence,  can  be  satisfied  by  existing 
(^atcvs  in  die  plan.  Sudi  effects  correspond  to  goals  that  do  not  require  adiievement  by  new 
(gators.  It  is  inqxwtant  not  to  insert  opaators  to  achieve  these  goals,  for  doing  so  implies 
that  actimi  is  required  when  it  is  not  In  HINGE,  there  is  a  data  structure  called  a  “phantom 
c^atm”  which  exists  mily  to  associate  a  goal  with  an  existing  operatm  whose  effects  h^pen 
to  establish  it  A  phantom  operator  is  simply  a  crmcrete  operator  whose  preconditinis  equal 
its  effects.  Hiantom  operators,  being  conaete.  may  form  the  basis  of  a  reduction  just  as  an 
instantiated  library  (^atm  sdiema  might  hiterprefing  a  plan  produced  by  HINGE  must 
include  removing  any  phantom  qieratms  and  redirecting  establishments  as  needed. 

3.422  Phantom  Goals.  While  phantom  qierators  are  an  artifact  of  HINGE 
planning,  phantom  goals,  cm  die  other  hand,  are  commmi  in  planning  domains  and  are  not 
unique  to  HINGE.  Phantom  goals  are  problem  goals  diat  ^pear  in  the  initial  state  and  will  not 
be  changed  by  the  plan  or  require  an  t^atw  to  adiieve.  Before  a  plan  is  produced  ( and  lacking 
special  domain  infnrmatioa).  it  is  mqiossible  to  tell  whedier  a  problem  goal  that  appears  in 
die  initial  state  is  a  phantom  goal  (will  remain  unchanged  by  the  plan).  Therefore,  possibly- 
idiantom  goals  present  some  difficulties  that  must  be  handled  during  search.  Section  3.7.2 
describes  a  deconqiosition  mediod  that  is  used  to  alter  seardi  based  on  the  presence  of  problem 
goals  that  possibly  are  phantom  goals. 

3.42  J  Serendipity  and  Reduction  Ordering.  A  concrete  qierattx'  in  a 
reduction  under  ctxisideratkm  can.  by  coinddoice.  establish  oae  cm:  more  preconditicms  for 
c^atOTS  in  the  plan  other  than  diose  established  by  its  parent  abstract  c^ator.  This  ability 
to  establish  extra  preconditicns  is  valuable  if  the  preccmditicxis  are  currenUy  established  by 
abstract  c^ieratcars.  If  so.  then  the  reduction  has  additional  merit  diat  should  be  considered  by 
the  planner.  HINGE  takes  advantage  of  diis  type  of  snendipity  as  described  in  Sectkni  3.4.3. 
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Table  1.  Cost  weights  for  wdering  reductions  by  means-everything  analysis. 


Reduction  descriptor 

META  Value 

Phanttxn  Operator  Reduction  (Existing  operators  achieve  goals) 
Serendipitous  Establishment  by  Operatm  (per  establishment) 
Abstract  Operatm  (per  abstract  qjeratm) 

Abstract  Operator  Effect  (per  abstract  operator  effect) 
Establishment  by  Abstract  Operator  (pm  establishment) 

Required  Ordming  Constraint  (per  anstraint) 

Preconditkms  (per  preconditimi) 

Effects  (per  effect) 

-10.000 

-500 

1,000 

500 

100 

50 

20 

10 

3.43  Ordering  Reductions:  Means-everything  Analysis.  To  control  search,  it  is 
heh)ful  to  assess,  in  as  much  detail  as  possible,  die  impact  of  inserting  a  particular  reduction 
into  the  plan.  Means-everything  analysis  is  a  domain-independent  extension  of  means-ends 
analysis  (Newell  and  Simon.  1963).  Fwr^ierators  in  the  reduction,  means-everything  analysis 
considers  not  (Xily  the  usefulness  of  die  primary  effects,  but  also  the  impact  of  side  effects  and 
preconditums.  In  particular,  means-everything  analysis  cmisiders 


•  die  ability  of  existing  cqieratKVS  to  satisfy  goals 

•  die  usefulness  of  (^atca  effects  for  serendipitously  satisfying  existing  goals 

•  die  potential  wwk  associated  with  unestabhshed  preconditions 

•  die  potential  fex'  backtracking  because  of  dq>endence  upon  planning  commitmenji 
(goals),  radier  dian  existing  operates 

•  the  potential  for  backtracking  because  of  required  ordering  cmistraints 

•  the  simplicity  of  die  operator  in  terms  of  die  number  of  its  preconditions  and  effects 

Means-everything  analysis  is  similar  in  spirit  to  the  domain-independent  heuristic  exten- 
sim  of  means-ends  analysis  in  SNLP,  but  means-everything  analysis  is  more  conqirdiensive 
in  terms  of  die  number  of  factws  that  it  ctxisiders.  In  HINGE,  means-everything  analysis  is 
implemented  as  a  metric  functum  mi  reductimis.  Table  1  shows  die  relative  values  assigned 
to  each  factor. 
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The  costs  shown  in  Table  1  were  chosen  based  on  extensive  problem  solving  expmence 
widi  HINGE.  During  problem  solving,  it  is  most  important  to  avoid  doing  any  work  that  has 
already  been  dtuie.  Iherefore,  a  reduction  containing  a  phantom  operatiM-  is  given  a  high 
negative  cost  because  phantom  operatixs  represent  outstanding  goals  which  can  be  achieved 
by  existing  plan  optf  attxs  and  require  no  additional  work.  Recognizing  reductions  tiiat  achieve 
goals  serend4>itously  is  also  imprxtant,  althcnigh  not  as  important  as  avoiding  work  tiiat  is 
already  drxie.  Therefore  reductions  that  support  serendipitous  establishments  are  awarded  a 
moderate  negative  cost  To  direct  search,  choosing  reductirxis  that  require  no  extra  wrxrk  is 
quite  inqiortant — more  inqxvtant  tiian  recogni^g  serendqiitous  goal  achievement  but  not  as 
inqxxiant  as  avoiding  work  that  is  already  dmie.  Furthermore,  tiie  amount  of  extra  wwk  is 
also  impixtant  Therefwe,  an  abstract  r^atcx-  in  a  reduction  adds  a  relatively  high  cost  and 
each  effect  m  the  abstract  (^atix*  adds  a  modo'ately  high  cost  Reductions  that  depend  on 
an  external  abstract  (^ator  for  (xie  (x  more  establishments  are  penalized,  but  not  as  much 
as  if  they  include  an  abstract  operator.  However,  if  a  axicrete  t^ator  can  establish  the  same 
precondititxi.  HINGE  always  prefers  tiie  concrete  q;)erat(xr  as  an  establish^^.  Each  ordering 
constraint  in  tiie  reductuxi  costs  something  because  ord^ing  constraints  inqiact  the  planner’s 
ability  to  insert  new  operatcxrs.  Ordering  constraints  are  not  considered  as  bad  as  dqiending 
<xi  an  external  abstract  opo'ator  to  establish  a  reductuxi  operator’s  precondition,  however. 
Finally,  the  precrxidituxis  and  effects  of  reducticxi  opo-ators  cost  a  small  amount  to  show 
prefvaice  fcx  sinqile  reductions  over  complicated  ones. 

Some  of  the  weights  are  designed  to  to  allow  one  factor  to  overwhelm  all  otiiers,  while 
otiier  weights  give  factors  mcxe  equal  crxisideratirxi.  For  exanqile,  a  reduction  with  a  phantom 
(^leratix  is  consulted  to  be  very  desirable  because  it  represents  knowledge  of  a  part  of  a 
sub-problem  tiiat  is  already  solved.  Therefore,  a  reduction  oxitaining  a  phantom  operator  is 
currently  preferred  in  any  case.^  In  ccxitrast,  if  two  reductions  both  axitain  only  one  concrete 

'^HINGE’S  reduction  metric  makes  no  assunqxioos  about  die  fonn  of  reductions  allowed.  Therefore,  it  is 
possiMediat  a  reduction  which  includes  many  abstract  operators  and  a  phantom  operator  would  not  be  prefeired. 
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(^)erator.  there  is  no  preference  between  diem  when  one  of  the  concrete  operators  has  two 
{Hrecondidons  and  (me  effect  while  the  other  operator  has  (me  precondition  and  four  effects. 

3J  Planning  in  HINGE 

HINGE  plans  by  reducing  abstract  operators  in  an  abstract  plan  until  there  are  no  more 
of  them  and  die  plan  is  concrete.  At  all  times.  HINGE  maintains  die  ^plicability  of  the  plan  by 
finding  reductitms  using  die  causal  model  implied  by  the  set  of  recorded  establishments  which 
are  part  of  the  VHPLAN.  It  may  be  helpful  to  some  readers  to  consider  a  simple  exan^ile  of 
generative  planning  at  diis  point 

Figure  15  shows  the  definitions  of  the  primitive  operattH*  schemas  for  the  blrmk-stacking 
domain.  The  move-to  operator  schema  repr^ents  the  actitm  of  moving  some  block  from  the 
top  of  (me  stack  to  the  t(^  of  another  stack.  Similarly,  the  stack  operator  schema  represents 
moving  a  block  from  the  table  to  a  stack  and  die  unstack  (^rator  schema  represents  moving 
a  bltmk  from  a  stack  to  die  table.  In  all  operatcM-  schemas,  variable  constraints  fierce  variables 
to  unify  only  widi  block  names  and  not  to  unify  with  the  table.  (Odierwise.  move-to  alone 
would  suffice),  figures  16-20  show  how  the  VHPLAN  emerges  as  HINGE  solves  Sussman’s 
Anomaly. 

HINGE’S  input  is  an  initial  plan,  consisting  of  operat(»s  init  and  final  which  are  made 
from  the  initial  state  and  problm  goals  acc(»’ding  to  the  definititm  of  initial  plan  givm  in 
Sectum  2.2.2,  and  a  library  of  (iperattH'  schemas  diat  may  be  used  to  crmstruct  reductions  of 
abstrard  operators.  Because  diis  example  is  fix’  generative  planning,  assume  diat  the  library 
contains  only  those  (qierator  schemas  shown  in  Figure  IS.  As  planning  begins,  HINGE  adds 
an  abstract  operator  as  die  root  of  die  plan  tree  and  adds  establishments  to  make  the  initial 
plan  applicable,  prtxhKting  the  VHPLAN  shown  in  Figure  16.  The  root  abstract  opiatin'  has 
no  precrmditions  and  has  effects  equal  to  the  problem  goals.  In  Figure  16,  die  effects  of  die 
root  abstract  opo'ator  have  been  used  to  establish  the  preconditions  of  final  to  make  the  pl^^n 
^plicable.  The  ordoing  (xmstraints  shown  in  Figure  16  exist  because,  by  definitirm, 
precedes  all  odier  opo-ators  while  all  otho-  operators  precede  final. 
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dMr(?-V) 

on(7-X.?-.Y) 

Fi^e  IS.  The  primitive  operator  schemas  for  the  block-stacking  domain. 


Figure  16.  The  initial  VHPLAN  for  HINGE  solving  Sussman’s  Anomaly. 


Figure  17.  The  VHPLAN  afta*  a  con^lete  decomposition  of  die  root  operator. 
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To  plan,  HINGE  recursively  selects  an  abstract  r^ratcM'  to  reduce,  annotates  the  ab¬ 
stract  operator  with  all  possible  reductions,  and  replaces  the  abstract  operahM-  with  a  selected 
reduction  to  produce  the  successive  plan.  Selecting  abstract  (gators,  finding  reductknis,  and 
choosing  a  reductuHi  to  replace  the  parent  abstract  q)eratOT  are  planning  decisions  that  are 
specified  in  Secticn  3.4,  although  fills  single  exanqile  will  not  illustrate  all  facets  of  search 
control  From  file  VHPLAN  shown  in  Figure  16,  HINGE  can  only  select  the  root  operator 
fw  reductkm.  Because  no  qierator  schema  in  file  library  can  be  instantiated  to  establish  the 
effects  of  the  root  qieratcM’,  the  only  reductions  must  be  decon^iositions.  Furfiiermore,  because 
file  (mly  decompositum  possible  for  an  abstract  operator  of  two  effects  is  a  con^lete  decom¬ 
position.  HINGE  finds  only  one  reduction,  the  complete  deconqiosition.  Reducing  the  root 
(qierator  with  its  complete  decomposititm,  adding  new  ordering  ctmstraints,  and  redirecting 
establishments  results  in  file  plan  indicated  by  file  VHPLAN  of  Figure  17. 

HINGE  must  now  select  between  reducing  abopl  and  eibop2.  Because  both  abstract 
c^ators  are  equally  deep  in  file  plan  tree  and  because  they  have  an  equal  number  of  effects,  the 
choice  is  arbitrary.  Suppose  HINGE  selects  abopl.  Two  operator  schemas  in  the  library  may 
be  instantiated  to  establish  file  effect  on(Afi).  The  schemas  are  named  move-to(?-X,?-Y)  and 
stack(?-X.?-Y).  Both  of  file  concrete  op&aXxxs  that  result  from  the  operator  schemas  require 
an  abstract  cqierator  to  help  establish  their  preconditions  and  therefore  both  of  the  resulting 
reductions  include  at  least  (me  ordering  ccmstraint  No  decompositions  are  possible  because 
abopl  has  <mly  <me  effect  Therefore,  fiiere  are  only  two  possible  reductions  of  abopl .  Neither 
reduction  has  clobbero's.  HINGE  selects  reductions  based  on  means-ev^ything  analysis  and 
the  heuristic  weighting  function  described  in  Section  3.4.3.  HINGE  selects  the  reduction  of 
stack(AJB)  because  it  has  fewer  preconditicms  that  cannot  be  established  by  existing  operators 
in  file  plan.  Thus,  the  abstract  operatcx*  required  to  partially  establish  stack(Afi)  has  fewer 
effects  fiian  file  one  required  for  move-to(A^).  The  relative  weights  make  this  factor  overwhelm 
anofiier  factor  fiiat  also  favors  the  reduction  of  stack(Afi)\  move-to(A,B)  has  (me  more  effect 
than  stack(A,B),  meaning  that  move-to(AJS)  is  more  complex  fiiat  stack(A^).  Figure  18  shows 
the  VHPLAN  that  results  after  HINGE  reduces  abopl. 
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Figure  18.  The  VHPLAN  after  reducing  abop  1. 


Figure  19.  The  VHPLAN  after  reducing  abop  2. 


Figure  20.  The  VHPLAN  representing  a  cono-ete  plan  resulting  from  reducing  abop  3. 
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HINGE  must  now  select  between  reducing  abopl  and  abop3.  Because  abop2  is  less  deep 
in  the  plan  tree  than  abop3,  HINGE  selects  abop2.  Once  again,  two  operator  schemas  in  the 
library  may  be  instantiated  to  establish  the  effect  on(B,C).  As  before,  die  schemas  are  move- 
to(?-X,?-Y)  and  stack(?-X,?-Y).  However,  this  time  the  concrete  operator  tiiat  results  from 
instantiating  stack(,  ?-X,?-Y)  does  not  require  an  abstract  operator  to  establish  any  preconditions , 
while  the  concrete  operator  that  results  from  instantiating  move-to(?-X,?-Y)  requires  an  abstract 
operator  to  establish  on(B,  ?-Z).  Once  again,  no  decompositions  are  possible,  and  there  is  a 
choice  between  just  two  reductions.  As  before,  neither  reduction  has  clobberers.  HINGE 
uses  means-everydiing  analysis  again  to  select  the  reduction  of  stack(B,C).  This  time,  it  is 
the  existence  of  an  abstract  operator  in  die  reduction  of  move-to(B,C)  diat  overwhelms  other 
cmisiderations  in  selecting  the  reduction.  HINGE  installs  the  reduction  of  stack(B,C)  in  the 
VHPLANby  making  it  the  child  of  abop  2  and  redirecting  ordering  establishments  as  shown  in 
Figure  19.  Notice  the  ordering  constraint  that  requires  stack(B,C)  to  come  before  stack(A^). 
HINGE  adds  this  ordering  constraint  to  preclude  stack(AJB)  from  clobbering  the  required 
establishment  <init,  clear(B),  stack(B,C)>. 

Finally,  HINGE  must  reduce  the  remaining  abstract  operator,  abop  3.  Two  operator 
schemas  in  the  library  may  be  instantiated  to  establish  die  effect  clear(A),  and  diere  are 
two  reductions  possible.  Hie  instantiated  op&ator  unstack(C)  requires  no  abstract  operator, 
so  HINGE  installs  a  reduction  containing  unstack(C),  resulting  in  the  VHPLAN  shown  in 
Figure  20.  Notice  die  ordering  constraint  diat  constrains  unstack( C)  to  come  before  stack(B,C). 
HINGE  adds  diis  ordering  constraint  to  preclude  stack( B,C)  from  clobbering  die  establishment 
Kinit.  clear(C),  unstack(C)>.  There  is  also  an  ordering  constraint  between  unstack(C)  and 
stack(Afi).  This  constraint  was  redirected  from  abop  3  when  it  was  reduced  by  unstack(C). 

The  current  plan  is  represented  by  the  leaf  nodes  of  die  plan  tree,  init,  mi  final.  To 
interpret  diis  plan,  die  leaves  of  the  plan  tree  are  interpreted  as  actirxis  and  the  ordering 
constraints  are  used  to  order  actions.  In  diis  exan^le,  the  plan  is  totally  ordered,  but  that  is 
not  goieraUy  true. 
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In  this  example,  no  backtracking  occurred  to  retract  a  poor  planning  decision.  However, 
in  the  event  that  an  abstract  operator  has  no  reductions,  HINGE  backtracks  chronologically  by 
removing  the  reduction  that  includes  the  abstract  operator  and  substituting  a  different  reduction 
associated  with  the  abstract  operator’s  parent  If  the  parent  has  no  more  reductions,  HINGE 
backtracks  until  some  ancestor  has  an  untried  reduction.  If  the  root  operator  is  reached  and  has 
no  more  reductions,  then  HINGE  reports  failure.  Alternatively,  if  no  more  abstract  operators 
remain  in  the  current  plan,  HINGE  returns  the  solution. 

3 .6  Hierarchical  Reuse  and  Efficiency 

HINGE  is  intended  to  promote  reuse  by  structuring  both  the  storage  and  reuse  oppor¬ 
tunities  of  learned  macros  hierarchically,  where  the  hierarchy  is  based  on  the  number  of  goals 
to  be  achieved.  The  number  of  goals  is  a  heuristic  indicator  of  the  amount  of  work  required 
and  hence,  the  search  that  may  be  saved  by  reuse.  In  addition,  the  number  of  goals  is  a 
feature  which  may  be  used  to  index  stored  reuse  candidates.  HINGE  actively  promotes  reuse 
of  the  most  powerful  candidate  available,  and  in  particular,  HINGE  seeks  reuse  candidates  in 
decreasing  order  of  the  number  of  goals  to  be  achieved.  Thus,  by  default,  HINGE  initially 
acts  like  a  case-based  planner  in  that  it  attenq>ts  to  solve  die  whole  problem  with  one  reuse 
candidate.  Oflierwise,  the  problem  is  decomposed  using  methods  described  in  Section  3.7  and 
the  subproblems  are  solved  through  recursive  calls  to  the  planner.  Eventually,  decomposition 
will  result  in  abstract  operators  that  each  represent  only  one  goal.  In  diis  event,  HINGE 
acts  like  a  generative  planner,  trying  to  solve  the  problem  using  only  primitive  operators. 
Using  tiiis  strategy  of  hierarchical  search,  HINGE  maximizes  the  likelihood  of  performance 
improvement  through  reuse  and  gracefully  degrades  to  generative  planning  whenever  reuse  is 
not  possible. 

There  is  a  subtle  difference  between  the  data  structure  used  in  the  HINGE  model  and 
a  goal  set,  but  the  difference  has  an  important  impact  on  search  control.  As  stated  in  Sec¬ 
tion  3.2.5,  the  biggest  improvement  of  HINGE’S  model  over  McAllester  and  RosenbUtt’s 
model  is  that  the  HINGE  algorithm  manipulates  a  data  structure  (abstract  operators  in  the 
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VHPLAN)  that  can  represent  a  subset  of  the  outstanding  goals  that  should  be  achieved  simul¬ 
taneously.  Because  this  set  of  goals  is  also  used  as  an  index  for  retrieving  a  reuse  candidate, 
the  HINGE  model  can  specify  a  desired  reuse  candidate.  Conversely,  a  set  representation 
for  goals  cannot  represent  a  distinguished  subset;  thus  for  models  that  depend  on  a  goal  set, 
finding  a  good  reuse  candidate  is  more  ht^hazard.  To  illustrate  this  point,  consider  the  block¬ 
stacking  problem  shown  in  Figure  21  and  the  two  macros  shown  in  Figure  22.  A  plaimer  that 
depends  on  a  goal  set  might  choose  to  insert  Macro  2  because  it  achieves  more  of  the  problem 
goals  fiian  does  Macro  1.  For  example,  STRIPS  would  select  Macro  2,  because  STRIPS  uses 
means-ends  analysis  and  a  difference  metric  based  on  the  number  of  goals  solved.^  Alterna¬ 
tively.  if  a  planner  can  represent  cUstmguished  subsets  of  the  outstanding  goals,  as  HINGE  can 
wifii  abstract  operators,  then  it  can  specify  fiiat  the  goals  on(AJB),  on(B,C),  on(CX>)  should 
be  solved  simultaneously,  and  therefore  that  Macro  1  is  preferable  to  Macro  2.  For  example, 
these  three  goals  are  represented  by  abop  i  in  the  VHPLAN  shown  in  Figure  23.  Macro  1  is 
a  oHtcrete  operator  that  achieves  all  effects  of  abop  1  and  serves  as  the  basis  of  a  reduction 
for  abop  1. 

Knowledge  of  which  subsets  of  the  outstanding  goals  should  be  achieved  simultaneously 
often  comes  from  the  domain,  rather  fiian  from  file  HINGE  planning  model.  For  example. 
relati(Hishq)s  between  objects  can  sometimes  be  used  to  identify  independent  sub-problems 
or  problems  fiiat  are  likely  to  be  independent  Here,  the  term  “independent”  inqilies  that  no 
harmful  goal  interaction  will  occur  in  the  plan.  In  Section  3.7,  for  exanqile,  two  goal  decom¬ 
position  strategies  which  are  based  on  a  domain-definition  of  independence  are  described.  If 
such  domain  knowledge  exists,  then  the  HINGE  model  can  represent  it  in  a  data  structure  and 
use  file  data  structure  to  direct  search. 

3.7  Decomposition  Methods 

In  the  HINGE  model,  if  no  appropriate  reuse  candidate  exists  for  a  particular  problem, 
fiien  file  plaimer  must  have  a  method  for  decomposing  abstract  operators  (partitioning  a  set 

^Means-eveiyfliing  analysis  would  pick  Macro  2  also. 
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Figure  22. 


Figure  23. 
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Figure  21.  A  block-stacking  problem. 
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Macros  that  could  be  used  to  solve  part  of  the  block-stacking  problem  in  Figure  2 1 . 


A  HINGE  VHPLAN  representing  sets  of  goals  to  be  solved  simultaneously  for 
the  r  'ock-stacking  problem. 
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of  goals).  There  are  0(a")  ways  to  deconspose  a  set  of  n  goals.  Therefore,  finding  a  good 
decomposition  by  exhaustive  search  may  be  expensive  depending  on  the  contents  of  the  library 
of  reuse  candidates  and  on  die  domain  (for  die  distribution  of  good  decompositions  in  the  space 
of  all  decompositions). 

One  apparent  method  of  goal  decomposition  is  complete-decomposition:  trying  to 
achieve  each  goal  by  itself.  This  method  precludes  reuse,  but  is  good  when  reuse  fails  and 
generative  planning  is  required.  Other  methods  of  decomposition  are  required  for  reuse.  In 
this  research,  several  methods  were  developed,  and  in  HINGE,  these  methods  are  ^plied  in 
a  particular  sequence  which  is  designed  to  maximize  reuse  opportunities  and  separately  solve 
independent  or  likely-independent  sub-problems.  The  developed  decomposition  mefiiods  are 
based  on  a  domain-dependent  definition  of  independence,  the  existence  of  possibly-phantom 
goals,  and  contents  of  the  operator  schema  library.  The  last  decon^x)sition  method  applied 
yields  a  complete  deconq)osition.  This  sequence  of  decompositions  gives  HINGE  its  graceful- 
degradation  behavior  and  guarantees  completeness  because  the  underlying  generative  planner 
in  HINGE  is  conq)lete.  The  following  sections  describe  different  methods  of  decomposition 
used  in  HINGE. 

3.7.1  Independence-based  Decomposition.  When  a  domain-definition  of  indepen¬ 
dence  exists,  independence  i  :  a  useful  basis  for  decomposition.  Operators  are  described  by 
propositimis  which  themselves  describe  fiie  relationshq)  between  objects  in  a  domain  (or  an 
attribute  value  for  an  object).  Often  a  relationshq)  between  two  objects  can  be  used  to  describe 
a  depotdence  between  them  that  is  important  from  a  planning  perspective.  For  example,  in 
Sussman’s  Anomaly  shown  in  Figure  7.  it  is  clear  fiiat  in  the  initial  state  Block  C  is  related 
to  Block  A  by  virtue  of  the  relation  on(C.  A).  By  analyzing  a  set  of  propositions,  sets  of 
related  objects  can  be  identified.  Because  a  state  is  a  set  of  propositions,  the  set  of  objects 
in  a  state  may  be  partitioned  so  that  each  partition-set  contains  objects  related  only  to  other 
objects  in  tiie  partition-set.  For  example,  the  objects  in  the  initial  state  shown  in  Figure  7  on 
page  20  are  {Block  A,  Block  B,  Block  C}  and  the  set  of  related  sets  of  objects  (the  partition) 
is  {  (Block  A,  Block  C}  (Block  B  }  } .  To  find  the  independent  objects  in  a  plan,  it  is  necessary 
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to  analyze  all  the  states  visited  when  the  plan  executes.  Alternatively,  it  is  possible  to  predict 
the  independent  objects  in  a  plan  by  analyzing  the  q)erat(KS  available  to  transition  between 
states.  For  example,  in  most  block-stacking  examples,  the  primitive  operator  schemas  are 
defined  so  fiiat  objects  that  are  independent  both  in  file  initial  state  and  in  die  problem  goals 
are  independmt  in  some  plans.  Because  of  fills,  file  independent  objects  can  be  used  to  define 
sub-problems  which  can  be  solved  independently. 

In  HINGE,  problems  which  cannot  be  solved  wifii  a  whole-problem  solution  are  de¬ 
composed  using  problem-independence  when  a  domain-definition  of  independence  exists.  As 
an  ^proximation  to  problem-independence,  HINGE  also  uses  goal-independence  which  is 
based  on  objects  that  are  unrelated  in  die  problem  goals. 

3.7J  Phantom-goal  Decomposition.  Sections  3.4.2. 1  and  3.4.2.2  described  phan¬ 
tom  operators  and  phantom  goals.  While  HINGE’s  phantom  operators  are  an  artifact  of 
HINGE  planning,  phantom  goals  or  at  least  “possibly-phantom”  goals  are  common  in  most 
planning  problems.  Any  proposition  fiiat  tqipears  in  the  initial  state  and  in  the  problem  goals 
is  possibly  a  phantom  goal.  Possibly-phantom  goals  arise  because,  before  planning,  it  is  hard 
to  predict  how  die  plan  will  affect  objects.  Therefore,  the  user  of  a  planning  system  generally 
includes  in  the  problem  goals  any  initial-state  proposition  that  should  be  maintained.  For 
example,  in  formulating  a  plan  for  deploying  troops,  one  problem  goal  might  be  to  have  trans¬ 
port  aircraft  engines  in  working  order  after  deployment,  even  though  the  engines  are  working 
initially.  The  planner  might  be  able  to  find  a  plan  that  does  not  affect  the  working  state  of  the 
aircraft  engines,  in  which  case  the  goal  is  a  phantom-goal,  but  if  not,  it  will  have  to  come  up 
with  a  plan  that  includes  repairing  die  engines  to  restore  fiiem  to  working  ord^. 

Possibly-phantom  goals  are  a  useful  basis  for  decomposition.  Any  goal  that  turns  out  to 
be  a  phantom-goal  requires  no  additional  operators  to  be  inserted  by  the  planner.  Therefore, 
it  is  often  a  good  strategy  to  separate  possibly-phantom  goals  from  other  goals  to  delay  their 
consideration.  In  HINGE,  the  set  of  possibly-phantom  goals  is  decomposed  completely,  and 
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the  abstract  (^)erator  hiat  represents  each  goal  is  artificially  deepened  in  the  plan  tree  to  delay 
its  consideratitHi. 

i.7J  Library-based  Decomposition.  Because  the  objective  of  decomposition 
methods  in  HINGE  is  to  increase  opportunities  for  reuse,  it  makes  sense  to  examine  the 
potential  ctmtribution  of  the  library  as  an  infcxinatimi  source.  After  all,  a  macro  operator 
schema  can  mily  be  reused  if  it  exists  in  fiie  library.  This  section  presents  two  methods  by 
which  knowledge  of  library  contents  can  be  used  to  find  a  decomposition.  An  advantage  of 
using  either  of  these  methods  is  that  retrieval  of  the  reuse  candidate  can  ht4)pen  at  the  same 
time  as  the  decomposition  and  without  additional  cost 

When  no  domain-dependent  notion  of  independence  exists,  tiiese  methods  can  be  useful 
However,  they  are  not  useful  fOT  inaeasing  tiie  size  of  a  library  which  contains  cmly  primitive 
operator  schema.  Clearly  no  reuse  candidates  can  be  retrievfti  if  the  library  has  none  to  begin 
with.  By  extension,  these  methods  are  less  likely  to  work  if  the  library  is  sparsely  filled  than 
if  it  oHitains  many  different  reuse  candidates. 

3.73.1  Big-chunk-decomposition.  In  planning  with  reuse,  the  best  perfor¬ 
mance  occurs  witii  reuse  of  tiie  macro  tiiat  saves  the  most  search.  Normally,  this  macro 
ctvresponds  to  the  one  that  solves  the  most  problem  goals.  When  the  library  has  no  macro 
(q)erator  schema  that  achieves  all  of  tiie  n  problem  goals,  the  best  alternative  is  to  find  a  macro 
(^lerator  schema  that  solves  n  —  1  of  the  problem  goals.  HINGE’s  metiiod  of  big-chunk- 
decomposition  uses  this  t^proach.  This  decon^xisition  method  requires  a  procedure  that 
deconqioses  n  goals  into  a  pair  of  sets,  one  with  n  —  s  goals  and  the  other  with  s  goals,  where 
s  =  1 , 2, 3,  ...S Limit.  In  HINGE,  all  such  pairs  of  sets  are  found  for  a  particular  value  of  s 
and  reuse  candidates  are  sought  to  achieve  goals  in  the  larger  set  of  tiie  pair.  If  no  candidates 
are  returned,  the  value  of  s  is  incremented  unless  its  limit  has  been  reached,  and  reuse  candi¬ 
dates  are  sought  again.  The  metiiod  fails  if  s  reaches  its  limit  before  a  reuse  candidate  is  found. 
Alternatively,  if  a  reuse  candidate  is  found,  HINGE  forms  a  decomposition  of  the  problem 
goals  based  on  tiie  reuse  candidate.  For  a  given  s,  HI  big-chunk-decomposition  yields 
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{n\)/{s\  *  {n  —  5)!)  pairs  of  sets  (so  large  values  of  n  and  S Limit  are  unlikely  to  give  good 
performance). 


3.732  Same-size-decomposition.  While  big-chunk-decmnposition  is  un¬ 
attractive  as  die  size  of  the  small  subset  ino'eases,  same-size-decomposition  is  a  polyno¬ 
mial  mediod  which  uses  the  library  to  deconq;)ose  a  planning  probleia  In  same-size- 
deconqxisition,^  reuse-candidates  are  selected  arbitrarily  from  the  library  and  tested  to  see 
if 


1.  diey  achieve  some  of  the  outstanding  goals  and 

2.  dieir  preconditions  can  be  established  by  existing  plan  operators. 

If  a  reuse  candidate’s  effects  are  a  subset  of  die  outstanding  goals  and  if  die  candidate’s 
preconditums  are  a  subset  of  any  of  die  possible  states  in  which  it  is  applied  (according  to 
operators  and  ordt^g  constraints  in  the  plan),  dien  the  reuse  candidate  solves  part  of  the 
problem.  The  solved  part  of  die  problem  and  die  remaining  part  indicate  a  decomposition. 

As  discussed  in  Sectkm  4.4.3.  the  subset  tests  require  0(n3)  time  in  die  wots!  case  for 
each  reuse  candidate  tested.  Therefrne,  in  the  wtxrst  case,  n  reuse  candidates  are  tested,  and  diis 
deconqiosition  mediod  requires  O(n^)  time.  However,  even  though  same-size-deconqxisition 
is  a  polynomial  mediod,  th<^  of  maaos  that  can  be  instantiated  from  library  operator 

sdienoas  is  quite  large.  For  (ius  reason,  good  poformance  is  not  likely  with  same-size- 
deconqiosition,  and  HINGE  does  not  curroitly  implement  same-size-decomposition. 

i.8  Learning 

3.8.1  Learning  Mechanism.  HINGE’S  learner  is  typical  of  odier  explanation-based 
learning  (EBL)  procedures  used  to  construct  macros  for  planning  systems.  To  make  a  macro 

^Hie  name  for  this  decomposition  method  is  indicative  of  its  (levelopiiieiit.iaflier  dan  its  usefiilness.  Because 
Ug-chunkdeconqxMitiaa  becomes  expensive  as  the  size  of  die  smaller  subset  increases,  another  decompositian 
method  was  sought  dut  could  decompose  proUems  more  evenly  at  lower  cost,  and  same-size-deconqwsitiom 
cansaoetimBsdodiis.  However,  same-size-decompositioa  may  be  used  to  produce  aibitnuily  lop-sided  decom- 
posttionsaswelL 
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from  a  plan,  all  that  is  needed  is  to  find  the  macro’s  preconditions  and  effects  and  give  die 
macro  a  name.  The  macro’s  preconditicais  are  the  set  of  all  plan  (q)erator  precondititms  which 
require  establishers  that  are  external  to  die  plan.  The  macro’s  effects  are  found  by  simulating 
die  plan  using  the  macro’s  preconditions  as  die  initial  case.  Prectmditions  and  effects  of  a 
macro  are  subsets  of  all  preconditions  and  all  effects,  respectively,  of  the  primitive  operators 
in  the  plan  represented  by  the  macro:  a  macro  is  annotated  with  this  plan,  but  the  plan  is  not 
used  during  planning  unless  the  macro  is  expanded.  In  many  macro  planning  systems  and  in 
HINGE,  a  learned  macro  is  generalized  (made  into  an  (^ator  schema)  fOT  con^iact  stcx-age. 

In  HINGE,  macros  are  learned  in  resp<»ise  to  problem  solving  without  regard  to  dieir 
usefulness  in  future  planning.  When  a  plan  is  returned,  HINGE  learns  all  sub-plans  diat  save 
search  wh^  tested  on  the  training  prtdilem.  Each  abstract  operator  in  the  plan  tree  is  die 
“root”  of  a  sub-plan;  the  sub-plan  consists  of  the  leaves  of  the  corresptmding  sub-plan  tree.^° 
After  ftlitninating  phantom  operators.  HINGE  learns  from  each  sub-plan  that  has  multiple 
concrete  (^atcHS  which  are  related  to  one  another.  As  discussed  in  Sectuxi  3.8.2,  plans  with 
unrelated  operatcH-s  are  not  generally  valuable  for  future  problem  solving,  and  learning  such 
plans  leads  to  potentially  large  storage  requiremmts.  Two  (gators  are  related  to  (me  another 
whenever  they  tqipear  together  in  an  establishment  or,  except  fra-  an  ordering  (xmstraint,  would 
clobber  an  establishment — in  short,  whenever  thwe  is  an  ordoing  constraint  between  them. 
Fck*  reascms  discussed  in  Secticm  3.8.2,  HINGE’S  learning  elemrat  uses  other  filters  as  well, 
but  n(me  of  die  filtering  criteria  depend  <m  a  predicticm  of  future  utility.  Therefore,  HINGE 
does  not  team  selectively,  as  PRODIGY  and  MORRIS  do. 

Kambhanqiati  and  Kedar  have  charact^ized  the  EBL  macro  learning  techniques  as 
being  unsuitable  for  teaming  plans  with  partially-ordered  operates  such  as  die  ones  teamed 
by  HINGE.  The  unsuitability  arises  because  reuse  of  a  generalized  solution  with  partially- 
(X’dored  ap&itors  may  result  in  variables  being  instantiated  in  a  way  diat  was  not  intended. 
For  example,  ccmsider  Kambhanqiati  and  Kedar’s  example  shown  in  Figures  24  through  27. 
Suppose  a  planner  returns  the  plan  with  partially-ordered  operators  illustrated  in  Figure  25  for 

sub-plan  nnist  be  a  oxicrete  plan,  and  ccoaete  (q)eratars  are  always  at  the  leaves  a  plan  bee. 
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the  problem  illustrated  in  Figure  24.  An  EBL  component  like  die  one  used  in  HINGE  returns 
die  generalized  plan  shown  in  Figure  26.  It  is  desirable  to  ensure  that  any  total  order  of  this 
partial  order  of  operattvs  will  solve  a  new  problem.  Now,  consider  using  die  generalized  plan 
<ni  die  problem  shown  in  Figure  27.  It  is  possible  to  instantiate  the  generalized  plan  to  solve 
the  new  problem  by  binding  die  variables  ?-W  and  ?-Z  both  as  B.  However,  this  plan  fails 
unless  a  particular  total  order  is  inqxised  on  the  operatm's.  Kambhampati  and  Kedar  think  diat 
the  generalized  plan  should  be  able  to  solve  die  new  problem,  but  doing  so  is  not  trivial.  They 
state 


To  avoid  this  problem,  the  EBG  [EBL]  alg(»ithm  needs  to  be  more  systematic 
in  accounting  to  all  possible  interacticms  among  operators  correspmiding  to  all 
possible  total  orders  consistent  with  the  partial  ordering.  There  are  two  options 
to  doing  this.  One  is  to  modify  the  algorithm:  F(»-  instance,  repeatedly  compute 
the  weakest  omdittois  of  all  total  orders  of  the  partial  t^der  and  dien  ctMijoin  diem 
in  some  way.  Another  ration  is  to  modify  the  input:  Provide  a  full  explanattoi 
of  carecmess  of  the  instantiated  partially  ordered  plan,  and  use  that  explanation 
to  produce  die  correct  generalized  initial  conditions  for  the  generalized  partially 
ffdered  plan.  (Kamtdiampati  and  Kedar,  1991) 

Kambhampati  and  Kedar  chose  the  second  ^proach.  They  present  an  interesting  and  complex 
technique  based  on  Quqiman’s  modal  truth  critnion  to  guarantee  that  generalized  plans  will 
be  applicable  despite  the  total  order  chosen. 

HINGE  solves  this  problem  much  more  simply  by  assuming  that  die  generalized  plan 
in  Figure  26  should  not  be  used  in  die  new  problem  in  Figure  27.  HlNGE’s  approach  to 
guaranteeing  that  generalized  plans  wwk  for  any  total  ordering  of  operators  is  to  use  variable 
s^araticm  ccaistraints.  For  example,  in  HINGE,  the  generalized  plan  in  Figure  26  would 
also  include  constraints  diat  die  variable  ?-W  cannot  be  instantiated  to  die  same  object  as 
is  the  variable  ?-Z  and  diat  the  variable  ?-X  cannot  be  instantiated  to  be  the  same  object  as 
the  variable  ?-Y.  In  HINGE,  even  for  generative  planning,  the  domain  theory  necessary  to 
prtqierly  instantiate  operatcH’  schemas  is  embodied  in  variable  separation  constraints  attached 
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AmbiMn  go*:  (cn(A.B).  an(C.D))) 


(only  oporMor  oohoma) 


Figure  24.  A  problem  for  a  partial-order  planner. 


Figure  25.  The  resulting  partially  wdered  plan. 


Figure  26.  The  resulting  generalized  plan. 


OmWattM) 

OxoUotngoak:  (an(A.B),  on(B,C)}) 

Hgure  27.  A  problem  for  which  the  generalized  plan  seems  ^propriate  but  fails  when  A  is 
put  on  B  first 
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to  operator  schemas.^^  Fot  example,  if  an  operator  schema  has  tiie  precondition  on(?-X,?-Y). 
dien  the  domain  the(H7  requires  that  that  ?-X  and  ?-Y  must  be  instantiated  to  a  different  objects. 
Using  variable  s^aration  crmstraints  to  restrict  die  possible  misuse  of  generalized  plans  is  a 
simple  solution  which  requires  no  new  algorithm.  This  simple  solution  avoids  much  work  at 
the  expense  of  some  generality  during  reuse. 

3.8.2  Learning  Filters.  Because  they  potentially  require  excessive  space  to  store  and 
because  diey  are  infrequendy  useful,  HINGE  does  not  learn  macros  whose  primitive  operators 
are  unrelated.  If  the  plan  represented  by  a  macro  M  has  any  two  primitive  operators  which  are 
unrelated,  dien  the  plan  has  independent  sub-plans  and  in  HINGE,  M  is  not  learned,  histead, 
HINGE  learns  any  of  the  independent  sub-plans  that  include  multiple  primitive  operators  and 
represents  these  sub-plans  as  macros.  Suppose  HINGE  learns  n  maaos  ctmresponding  to  die 
independent  sub-plans  widi  multiple  primitive  operators.  Now  consider  diis  question:  why 
should  HINGE  learn  only  the  n  macros  and  not  M?  The  answer  is  that  there  are  potentially 
a  large  number  of  maaos  like  M  that  represoit  plans  with  independent  sub-plans  and  these 
macros  are  unlikely  to  be  reused.  Ihere  are  potentially  a  large  number  of  macros  like  M 
because  the  n  macros  of  independent  sub-plans  (and  die  other  macros  like  them  already  in  the 
library)  may  be  combined  in  an  enonnous  number  of  ways.  Each  of  these  combinations  solves 
a  specific,  large  problem,  and  the  probability  of  encountering  this  problem  is  small  (assuming 
a  somewhat  uniform  distribution  of  problems).  Rather  than  learn  and  reuse  macros  like  M. 
HINGE  relies  (m  deconqxisition  mediods  described  in  Section  3.7  to  break  up  large  problems 
into  their  independent  sub-problems  which  can  be  solved  with  known  macros. 

HINGE  also  filths  any  macro  that  does  not  appear  to  be  useful  for  saving  search. 
Including  a  macro  in  the  set  of  candidates  ctmsidered  expands  the  search  space.  In  some 
cases  sudi  as  when  die  maao  is  small,  the  macro  does  not  save  enough  search  to  pay  for  the 
expanded  search  space  it  creates.  To  identify  these  unproductive  macros,  HINGE  tests  die 

hinge  opentora  have  mofe  attributes  dian  a  name.  iHccooditiaiis.  and  efiects.  These  additional  attributes 
are  useful  in  the  planning  process,  but  diey  are  not  fundamental  to  describe  planning  models  or  characteristics  of 
plans.  Therefore,  these  attributes  have  not  been  described  to  simplify  die  discussion. 
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training  problem  to  see  if  having  the  macro  operator  schema  in  the  library  results  in  faster 
planning  tfian  not  having  it  in  the  library.  If  perframance  does  not  improve,  the  macro  is 
discarded.  Importantly,  this  filter  is  not  based  on  the  relative  amount  of  %arch  saved;  instead, 
all  macros  that  save  search  on  the  training  problem  are  learned. 

The  learner  in  HINGE  also  has  a  filter  to  keep  duplicate  macro  operator  schemas  out 
of  die  library.  Learning  duplicate  concepts  does  not  affect  the  function  of  the  library  or 
macro  operator  schemas  retrieved  from  it  during  die  course  of  planning.  However,  search- 
space  expansitm  and  macro  testing  costs  are  minimized  when  duplicate  operator  schemas  are 
avoided. 

Unlike  MORRIS ,  PRODIGY,  and  similar  problem  solvers  that  rely  on  selective  learning. 
HINGE  learns  all  search-saving  maaos  that  cmrespixid  to  independent  problems.  HINGE 
uses  filters  to  avoid  learning  useless  macros,  but  it  does  not  use  filters  based  on  utility  in  future 
planning  F(h’  example,  PRODIGY  uses  a  selective  filter  based  on  (Mintcm,  1990): 

1.  a  prediction  about  the  relative  probability  that  macros  will  be  used  in  future  problem 
solving 

2.  the  relative  amount  of  search  saved,  and 

3.  the  relative  cost  of  retrieval. 

HINGE  uses  none  of  these  as  filter  criteria  because  HINGE  does  not  learn  selectively. 

3S^  Reuse  allowed  in  HINGE 

HINGE’S  plan-space  planning  model  is  primarily  responsible  for  HINGE’s  ability  to 
reuse  maaos  more  flexibly  than  state-space  maao  planners  like  MORRIS.  As  discussed  in 
Section  2.4.3,  HINGE,  like  SNLP+EBL,  can  insert  maao  operators  anywhere  in  die  plan. 
Kambhanqiati  and  Chen  have  concluded  that  die  ability  to  insert  macros  anywhae  in  the  plan 
allows  reuse  precluded  by  state-space  plannas  (Kambhampati  and  Chen,  1993).  HINGE  and 
SNLP+EBL,  developed  at  about  the  same  time,  wae  the  first  macro  planners  with  this  ability. 


Bodi  SNLP+EBL  and  HINGE  can  expand  two  macros  and,  ordering  constraints  per¬ 
mitting.  interleave  the  corresponding  primitive  operates  in  order  to  solve  a  problem.  This 
ability  allows  HINGE  and  SNLP+EBL  to  reuse  macros  that  would  otherwise  conflict  with  one 
another.  (Kambhanqiati  and  Qien,  1993) 

Like  other  macro  planners.  HINGE  also  has  a  more  flexible  reuse  policy  than  case-based 
planners  which  allow  only  one  reuse  candidate  and  will  not  reuse  candidates  below  a  certain 
size.  Unlike  case-based  planners,  HINGE  can  decompose  a  set  of  goals  which  caimot  be 
achieved  directly  and  reuse  mult4>le  solutions  from  previously  encountered  problems. 

Unlike  other  macro  planners,  HINGE  explicitly  represents  certain  subsets  of  the  out¬ 
standing  goals  to  direct  the  search  for  reuse  candidates.  This  ability,  supported  by  decom¬ 
position  methods,  allows  HINGE  to  specify  and  find  good  reuse  candidates  much  faster  than 
when  outstanding  goals  are  shved  in  a  set 

3.10  Summary 

HINGE  is  a  macro  planner  whose  flexible  macro  insertion  capabilities  and  unique 
search  strategy  promote  reuse  and  facilitate  learning  for  improved  planning  performance. 
HINGE  uses  abstract  operators  to  represent  a  set  of  goals  to  be  solved  simultaneously  in  the 
plan.  The  overall  search  control  in  HINGE  is  depfli-first  with  chronological  backtracking. 
To  focus  attention  on  high-level  goals,  abstract  operators  are  selected  based  on  their  depfli 
in  the  plan  tree.  HINGE  seeks  reuse  candidates  that  simultaneously  achieve  the  goals  of 
abstract  (q^erators.  When  no  such  candidates  are  possible,  HINGE  decomposes  the  goals 
represented  by  the  abstract  operator  to  maximize  the  opportunities  for  reuse,  eventually  solving 
the  problem  generatively  when  no  reuse  is  possible.  In  this  way,  HINGE  reuses  the  most 
powerful  macros  first  and  retains  completeness.  HINGE’S  conventional  learning  module  was 
made  suitable  for  partial-order  planning  by  the  relatively  simple  method  of  posting  variable 
separation  constraints,  rath^  than  die  complex  method  proposed  by  Kambhampati  and  Kedar 
(Kambhampati  and  Kedar,  1991).  By  virtue  of  its  plan-space  planning  model,  HINGE  and 
SNLP+EBL  are  the  first  macro  planners  that  can  solve  any  problem  by  reusing  available 


macros,  even  when  those  macros  must  be  expanded  and  interleaved  to  integrate  sub-problem 
solutions. 
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IV.  Addressing  the  Utility  Problem 


4.1  Introduction 

The  utility  problem  is  the  possibility  that  a  problem  solver  which  learns  and  reuses 
solutions  will  perform  worse  with  additional  learning.  Previous  research  has  focused  on  some 
of  the  costs  associated  with  reusing  plans.  In  my  research,  I  treat  the  utility  problem  using 
an  eclectic  approach,  paying  attention  to  both  costs  and  benefits  of  reusing  plans.  Chapter 
m  described  HINGE  and  the  features  of  HINGE  that  promote  beneficial  reuse  such  as  its 
plan-space  planning  model  and  its  hierarchical  search  method.  This  chapter  focus  only  on 
utility  problem  costs.  In  particular,  this  ch^ter  describes  two  methods  that  strictly  contain 
the  costs  of  reuse:  an  efficient  retrieval  method  and  a  policy  of  selective  reuse.  The  chapter 
begins  by  analyzing  the  utility  problem  and  previous  planning  methods  that  motivate  selective 
reuse.  Section  4.3  defines  selective  reuse  and  an  important  implication  of  selective  reuse. 
Because  selective  reuse  depends  on  efficiently  finding  a  specific  reuse  candidate  in  the  library. 
Section  4.4  desaibes  the  development  of  a  discriminating  retrieval  method  that  takes  advantage 
of  complete  problem  information.  Section  4.5  shows  that  selective  reuse  limits  both  search 
and  the  expansion  of  the  search  space  better  than  other  reuse  policies.  Section  4.6  shows  how 
selective  reuse  can  be  relaxed  to  solve  problems  by  decomposition  using  reuse  candidates 
which  are  likely  to  be  appropriate  and  to  require  no  future  work. 

4.2  Analyzing  the  Utility  Problem 

4.2.1  Separate  Costs.  The  which  may  be  separated  into  these  components: 

1.  the  cost  of  retrieving,  from  the  library,  reuse  candidates  that  are  likely  (or  guaranteed) 
to  be  appropriate  (i.e.,  to  be  on  a  solution  path). 

2.  the  cost  of  selecting  from  among  the  retrieved  reuse  candidates 

3.  the  cost  of  doing  future  work  which  might  arise  from  trying  to  reuse  the  best  candidate; 
this  cost  may  be  further  broken  down: 


71 


(a)  if  the  selected  reuse  candidate  is  appropriate,  there  is  only  the  cost  of  search 
required  to  establish  any  of  the  candidate’s  preconditions  that  are  not  established 
by  existing  plan  operators:  otherwise. 

(b)  if  the  selected  reuse  candidate  is  inappropriate,  there  is  the  cost  of  search  required 
to  discover  that  its  preconditions  cannot  all  be  established  plus  the  cost  of  finding 
and  testing  an  appropriate  reuse  candidate. 

4.2.2  Previous  Methods  for  Containing  Costs.  As  described  in  Qiapter  n,  selective 
learning  has  been  the  method  of  choice  for  containing  the  utility  problem.  Selective  learning 
limits  retrieval  cost  directly  by  constraining  die  size  of  the  library  (and  therefore,  die  search 
required  to  find  a  particular  reuse  candidate  in  the  library).  By  limiting  the  number  of  reuse 
candidates  available,  selective  learning  also  limits  the  expansion  of  the  search  space  to  some 
extent.  However,  the  primary  motivation  for  selective  learning  is  limiting  the  retrieval  cost 

In  the  literature,  retrieval  has  been  characterized  as  requiring  general  search  and  match¬ 
ing.  Because  matching  and  search  are  supposedly  required,  the  cost  of  retrieval  has  been 
claimed  to  grow  exponentially  with  both  the  size  of  the  learned  macros  and  the  number  of 
them  in  a  library,  despite  any  indexing  mediod  used  (Minton,  1990).  However,  these  charac¬ 
terizations  are  not  necessarily  true.  For  example,  it  is  possible  to  find  a  polynomial  algorithm 
for  retrieving  expropriate  macros,  as  shown  in  Section  4.4. 

(Zhapter  n  also  described  case-based  methods  which  address  the  utility  problem.  Case- 
based  planners  usually  have  efficient  retrieval  methods  that  do  not  require  a  general  search 
algorithm.  Therefore,  case-based  planners  do  not  depend  on  selective  learning  to  retrieve  a 
reuse  candidate  quickly.  To  contain  the  expansion  of  the  search  space  caused  by  consider¬ 
ing  additional  operators,  case-based  planners  typically  limit  reuse  to  a  single  macro  which 
is  modified  to  solve  the  current  problem.  There  are  three  drawbacks  associated  with  this 
^proach: 

1.  the  retrieved  reuse  candidate  is  only  heuristically  similar  to  the  desired  plan;  there  is  no 

guarantee  of  appropriateness. 


72 


2.  allowing  only  one  reuse  candidate  means  that  new  problems  solvable  using  case-based 
plaiming  are  those  which  are  similar  to  previously-solved  problems,^  and 

3.  it  is  impossible  to  predict  the  amount  of  additional  work  required  of  the  planner  to 
modify  the  reuse  candidate  so  that  it  is  a  solution  to  the  current  problem. 

4J  Selective  Reuse 

Considering  the  costs  associated  with  plan  reuse  (Section  4.2.1),  the  best  method  of 
containing  costs  is  to  retrieve  only  ^propriate  reuse  candidates  so  long  as  the  cost  of  doing 
so  is  not  too  high.  If  the  retrieved  reuse  candidates  are  all  guaranteed  to  be  appropriate,  then 
selecting  one  of  diem  costs  virtually  nothing  because  the  choice  is  arbitrary;  inserting  any 
one  of  diem  into  a  plan  will  lead  to  a  solution.^  The  future  work  entailed  by  inserting  an 
appropriate  operator  is  the  work  done  by  the  planner  to  establish  any  of  its  preconditions  that 
caimot  be  established  by  existing  plan  operators.  In  contrast,  if  an  inappropriate  operator 
is  inserted  into  the  plan,  then  its  insertion  entails  both  the  work  of  establishing  operator 
prectMiditions  (until  one  is  found  diat  cannot  be  established)  and  the  additional  work  done  by 
die  planner  to  backtrack  and  find  an  appropriate  operator.  In  general,  the  future  work  entailed 
by  inserting  either  type  of  reuse  candidate  cannot  be  predicted. 

If  retrieving  only  tqipropriate  reuse  candidates  is  the  best  method  of  containing  the  costs 
of  plan  reuse,  dien  how  can  r^.oropriate  reuse  candidates  be  identified?  Assuming  no  special 
domain  knowledge  for  identifying  expropriate  reuse  candidates,  diere  are  only  two  ways  to 
identify  appropriate  reuse  candidates.  The  first  way  is  to  test  them  by  inserting  them  into  a 
plan  and  using  the  planner  itself.  If  inserting  a  reuse  candidate  into  a  plan  does  not  cause 
backtracking  during  the  search  for  a  solution,  fiien  the  reuse  candidate  must  have  been  on  a 
solution  path,  Le.,  apprc^nate.  Clearly,  this  test  is  of  no  value  because  it  requires  finding  a 

^  VcIoso’sFrodigy/AnaIjOGY  allows  multiple  reuse  candidates.  However,  because  it  depends  on  a  decompo¬ 
sition  medud  similar  to  l^-chunk-deconqxnitiQn  (Sectitm  3.7.3).  Frodigy/Analogy  cannot  efficiently  solve 
problems  ffiat  must  be  decomposed  into  same-size  chunks.  Therefore,  in  some  sense  PRODIgy/Analogy  is  also 
limited  to  solving  problems  that  are  similar  to  problems  solved  previously. 

^Of  course,  selecting  one  qypropriate  reuse  candidate  fiom  a  set  of  them  could  be  based  on  another  criterion 
such  as  execution  cost  or  side-efiect  preference. 
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solution;  the  appropriateness  of  a  reuse  candidate  is  needed  before  it  is  inserted  into  a  plan, 
not  whoi  the  planner  solves  the  problem. 


The  second  way  to  identify  appropriate  reuse  candidates  is  to  use  a  description  of  the 
problem  solved  by  the  reuse  candidate  and  compare  it  with  the  current  planning  problem.  The 
reuse  candidate  solves  the  current  planning  problem  if 

1.  die  effects  of  the  reuse  candidate  achieve  all  problem  goals,  and 

2.  the  preconditions  of  the  reuse  candidate  are  a  subset  of  the  initial  state 
or,  in  plan-space  terms,  if 

1.  the  effects  of  the  reuse  candidate  can  establish  the  preconditions  of  final,  and 

2.  all  of  the  preconditions  of  the  reuse  candidate  can  be  established  by  init. 

Any  reuse  candidate  that  meets  these  criteria  solves  the  whole  problem  and  is  therefore  called 
a  “whole-problem  solution.”  Inserting  such  a  reuse  candidate  into  a  plan  clearly  leads  to  a 
solution,  thus  such  a  reuse  candidate  is  appropriate  by  definition.  Because  there  is  no  general 
method  for  predicting  the  appropriateness  of  reuse  candidates  that  are  not  whole-problem 
solutions,  the  following  result  holds: 

Synonymous  Terms:  In  die  context  of  a  particular  planning  problem,  an 

appropriate  reuse  candidate  means  the  same  thing  as  a  whole-problem  solution.^ 

There  are  two  important  implications  that  arise  because  ^propriate  reuse  candidates  are 
also  whole-problem  solutions.  First,  it  becomes  quite  clear  how  ^propriate  reuse  candidates 
can  be  retrieved  from  a  library  and  what  information  is  required  for  retrieval.  Reuse  candi¬ 
dates  are  described  by  die  problem  they  solve;  die  initial  state  is  represented  by  the  macro’s 
preconditions  and  the  problem  goals  achieved  by  die  macro  are  represented  by  its  effects. 
Retrieving  a  candidate  requires  an  index  that  captures  macro  preconditions  and  effects  and 

^The  HINGE  planning  modd  assumes  dut  no  domain  knowledge  exists  to  guarantee  the  appropriateness  of 
a  reuse  candiriate.  Another  planner  may  not  malm  this  assurrqrtioa.  so  these  two  terms  may  not  by  syruuymous 
in  die  contmit  of  diis  odier  planner. 
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this  same  index  nuist  be  generated  by  a  new  problem  which  the  macro  can  solve.  To  generate 
the  right  indexing  function,  con:q)lete  information  about  the  new  problem  is  required.  That  is, 
its  initial  state  and  problem  goals  must  be  completely  specified.  With  these  conditions  and  the 
appropriate  index,  efficirat  retrieval  is  possible.  Sections  4.3.1  and  4.4  address  indexing  and 
develop  an  efficient  retrieval  method  for  finding  only  appropriate  reuse  candidates. 

The  second  implication  from  the  equivalence  of  “^propriate  reuse  candidate"  and 
“whole>problem  solution"  is  that  the  following  sinq)le  result  holds: 

Future  Work  Required  by  Appropriate  Macros:  Inserting  an  appropriate 
reuse  candidate  (a  whole-problem  solution)  into  a  plan  entails  no  future  plaiuiing 
work. 

The  costs  associated  with  the  utility  problem  are  contained  file  most  when  reuse  is 
confined  to  only  appropriate  reuse  candidates.  This  is  the  basis  of  selective  reuse. 

Selective  Reuse:  The  policy  of  retrkving  and  inserting  only  one  appropriate 
reuse  candidate  into  a  plan. 

Clearly,  if  selective  reuse  only  allows  reuse  of  whole-problem  solutions,  then  it  is  not  inter¬ 
esting  when  a  whole-problem  solution  does  not  exist  in  the  library.  However,  a  relaxed  form 
of  selective  reuse  can  be  applied  to  sub-problems  which  result  from  deconqxising  a  planning 
problem.  Tbis  relaxed  form  of  selective  reuse  uses  incomplete  problem  inftxmation.  but 
assumes  fiiat  file  information  is  complete.  Hie  method  is  described  in  Section  4.6.  Before 
analyzing  selective  reuse  fiirdier,  it  is  important  to  describe  the  retrieval  method  which  makes 
selective  reuse  a  viable  method  of  controlling  the  utility  problem. 

43.1  “Impedance  Mismatch”  in  an  Index.  Part  of  the  difficulty  in  finding  a 
dismminating  index  even  with  sufficient  information  is  that  fiie  problem  solver  and  learner 
often  have  different  "views"  of  the  information.  These  differrat  views  lead  to  a  potential 
"impedance  mismatch"  when  making  the  index  fw  a  particular  ctmcqiL  That  is,  the  index  that 
file  learner  uses  to  sttve  the  conc^t  may  be  quite  different  from  the  index  that  the  problem 
solvo*  will  use  to  try  to  retrieve  the  same  concept  This  effect  is  easy  to  see  in  maao  planners. 
A  maao  often  has  fewa  preconditiais  than  the  initial  state  of  a  problem  that  the  maao  solves 
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(if  so,  dien  part  of  the  initial  state  is  not  required  to  establish  preconditions  in  the  macro). 
Therefore,  an  impedance  mismatch  results  if  the  index  tries  to  match  the  initial  state  with 
preconditions  of  reuse  candidates.  Impedance  mismatch  is  a  recognized  problem  in  other 
EBL  systems  also  (Keller.  1987,  Minton.  1990).  Minton  notes; 

“...as  traditimially  viewed,  die  operationality  criterion  does  not  take  into  account 
how  the  learned  descrqition  will  be  used  later  to  in^rove  the  performance  [of 
the]  system,  which  determines  its  benefit  (Minton.  1990).’’ 

To  avoid  impedance  mismatch  in  constructing  an  index,  a  careful  analysis  of  the  domain  must 
be  used  to  find  ways  to  translate  problem  descr4)tions  and  leamed-soludon  descriptions  into 
identical  indices.  In  Section  4.4,  such  an  analysis  is  described  for  die  block-stacking  domain. 
Given  an  tqipropriate  index,  the  polynomial  retrieval  mediod  described  in  Section  4.4.3  may 
be  used. 

4.4  Retrieval:  An  Extended  Example 

This  section  presents  an  extmded  example  that  develops  an  index  and  polynomial 
retrieval  method  based  on  selective  reuse.  SecdtHi  4.4.1  describes  an  indexing  scheme  for 
linear-ordo'  retrieval  of  maao  operators  that  completely  solve  problems  in  a  cmistrained 
STRIPS  block-stacking  planning  problem.  Unlike  general  search  and  matching  methods,  die 
indexing  and  retrieval  mediod  does  not  require  an  exptxiential  amount  of  dme  to  find  macros 
despite  the  size  of  die  macros  or  the  number  learned.  This  exanqile  shows  that  retrieval  does 
not  require  general  (exponential)  search  when  conqilete  problem  informadon  exists,  and  only 
constrained  search  is  required  when  inconqilete  problem  infcMmadon  exists. 

In  Secdon  4.4.2,  die  effects  of  removing  some  of  the  assumptions  in  the  example  problem 
are  discussed  ak»g  widi  the  space/dme  trade-offs  that  exist  As  shown  in  Secdtm  4.4.3,  in 
die  case  that  die  impedance  mismatch  cannot  be  eliminated,  a  polynomial-time  algorithm 
may  exist  fos  finding  a  reuse  candidate  that  is  guaranteed  to  be  apprqiriate.  In  Section  5.3, 
en^irical  results  are  presented  that  suggest  problem  solving  perfcxmance  degrees  linearly 
with  the  size  of  the  library  in  die  worst  case. 
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Figure  28.  The  retrieval  process  for  a  macro  planner. 


4.4.1  Solving  the  Utility  Problem  in  a  Blocks  World.  Figure  28  shows  the  retrieval 
process  of  a  macro  plann^.  A  t  the  beginning  of  the  planning  process,  die  planner  has  a  set  of 
goals  to  be  achieved  and  a  specification  of  die  initial  state.  Using  this  information,  a  retrieval 
index  function  returns  an  index  diat  is  used  to  retrieve  a  set  of  macro  qierator  schemas  (plan 
schemas)  that  might  achieve  die  goals,  given  the  current  state.  The  retrieved  macro  operator 
schemas  are  tested  until  an  instantiation  of  one  of  them  solves  the  problem.  If  none  of  die 
macro  operator  schemas  can  be  used  to  solve  die  problem,  then  die  macro-based  planner  fails 
and  a  solution  must  be  generated  from  primitive  operators.  Whenever  such  a  failure  occurs, 
planning  widi  reuse  takes  Itmger  dian  generative  planning.  On  the  other  hand,  if  it  can  be 
guaranteed  that  all  retrieved  maao  operator  schemas  can  be  instantiated  to  solve  die  problem 
(le.,  the  index  is  sufficiently  discriminating  tt>  find  appropriate  reuse  candidates),  then  no 
testing  is  required,  and  failure  of  the  retrieval  mechanism  bears  only  a  small  cost  Widi  the 
best  hashing  mediods,  die  worst-case  time  (vder  of  retrieval  is  0{n)  in  the  niunber  of  stix’ed 
macro  qierator  schemas  (Horowitz  and  Sahni,  1990).  Therefore,  the  time  complexity  of  the 
best  retrieval  algoridim  is  no  worse  dian  linear  in  the  number  of  macro  oper^or  schemas. 

If  the  macro  planner  fails,  a  conqitete,  generative  planner  produces  a  plan  diat  achieves 
the  goals,  given  the  initial  state.  Whenever  a  plan  is  generated,  a  generalization  module  creates 
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die  macro  (^lerator  schema  by  substituting  variables  for  objects  and  object  attribute  values. 
A  storage  index  functitm  is  used  to  store  die  new  macro  operator  schema.  In  order  to  find 
an  indexing  scheme  that  overcomes  the  utility  problem,  it  is  necessary  to  specify  the  index 
functions  used  to  stme  and  retrieve  macro  c^ierator  schemas  and  show  diat  the  retrieval  index 
allows  retrieval  of  only  those  schemas  that  can  be  instantiated  to  solve  a  particular  planning 
problem.  This  is  demonstrated  below  with  a  tighdy  constrained  version  of  the  block-stacking 
domain. 

In  this  ccHistrained  block-stacking  domain,  six  blocks  may  be  arranged  only  in  the 
configurations  shown  in  Figure  29:  this  will  hold  for  initial,  goal,  and  intermediate  states. 
Also,  die  position  of  stacks  on  the  table  is  part  of  the  state  description.  Each  of  the  11 
configuratimis  may  be  arranged  in  6!  different  ways,  so  the  total  number  of  states  for  diis 
world  is  7, 920.  There  are  62, 726, 400  (7, 920^)  state  transitions  (7, 920  of  diem  “null” 
transitions)  and  an  infinite  number  of  plans  possible  for  each  state  transition.  The  primitive 
operators  are  defined  to  perform  diese  transitkxis  in  a  single  step.  One  more  simplifying 
assumption  is  required  to  find  a  completely  disorimmating  indexing  scheme:  die  set  of  goals 
provided  to  the  planner  specifies  the  goal  state,  rath^  than  a  set  of  propositions  die  goal 
state  must  include.  In  odier  wtxds,  a  problem  is  described  by  an  initial  state  and  a  goal  state 
radier  than  an  initial  state  and  a  set  of  problem  goals.  The  initial  and  goal  states  describe 
the  planning  problem  and  also  describe  die  “interface  characteristics”  (I/O)  of  plans  derived 
to  solve  the  planning  problem;  hence,  diere  will  be  no  unpedance  mismatch.  Similarly,  a 
graeralization  of  die  two  states  defines  a  generalized  planning  problem  and  also  describes  die 
interface  characteristics  of  a  generalized  plan  for  solving  it  Therefore,  die  retrieval  index 
fimctkHi  m^  a  planning  problem  into  a  generalized  planning  problem  index,  and  the  storage 
index  functuxi  returns  an  index  fcx  a  generalized  planning  problem. 

Figure  30  (a)  shows  an  exanqile  of  a  transition  betwe«i  a  generalized  initial  state  and  a 
generalized  goal  state.  Hgure  30  (b)  shows  the  same  transiti<Mi  widi  a  diffnent  set  of  variable 
assignments.  Fw  space  efficimcy.  die  index  functuxi  should  m^  one  transition  pattern  into 
a  single  index,  regardless  of  the  variable  names.  One  way  to  do  so  is  to  label  the  initial  state 
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Figure  29.  Allowed  block-stacking  configurations  and  initial  state  variable  labeling 
convention. 

with  variables,  tising  the  ctmvention  shown  in  Figure  29.  Figure  30  (c)  shows  the  appropriate 
transition  rq)resentation  witii  ttus  conventicm. 

Now  die  index  functitms  can  be  specified.  For  learning,  the  storage  index  function  maps 
a  transition  between  geno'alized  states  into  two  6x6  arrays  whose  elements  are  either  empty  or 
contain  the  name  of  a  variable  representing  a  block.  Figure  31  shows,  in  graphical  form,  the 
resulting  index  fcx  die  macro  represented  by  die  transition  shown  in  Figure  30.  For  planning, 
die  retrieval  index  functum  generalizes  the  initial  and  goal  states  and  dien  ^plies  die  storage 
index  function. 

As  in  many  problems,  there  can  be  an  infinite  number  of  plans  possdile  for  each  of 
die  state  transititms  (albeit  most  of  diem  are  inefficient).  However,  for  retrieval,  (xily  the 
generalized  interface  characteristics  are  of  concern.  In  the  example  world,  there  are  only 
87, 120  (62, 726, 400/6!)  transiticxis  between  geno’alized  states  because  each  transition  may 
be  instantiated  6!  ways.  Thus,  graeralizatitm  saves  considerable  space  (for  a  small  cost 
in  time)  by  mqiping  the  given  problem  to  a  goi^alized  problem  and  then  instantiating 
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Figure  30.  Alternative  variable  labeling  for  block-stacking  transitk»is. 


Figure  31.  An  index  for  die  macro  operator  schemas. 


the  retrieved  maao  qierator  schemas  to  produce  operators.  Furthermore,  if  a  sufficiently 
disaiminadng  index  is  available,  there  is  no  cost  of  testing  macro  operator  sdiemas  for 
applicability.  When  a  set  of  schemas  is  stored  under  an  index  oHTesponding  to  a  single 
transiticMi.  the  implication  of  using  a  sufficiratly  discriminating  index  is  profound:  all  the 
schemas  are  retrieved  simultaneously  and  are  guaranteed  to  be  ^prt^riate.  No  testing  is 
required,  selection  amcmgst  them  can  be  based  upon  other  criteria  (such  as  executitxi  cost), 
and  the  retrieval  of  the  multiple  plans  does  not  add  to  the  cost  of  reuse. 


4,42  Expanding  the  Blocks  World.  The  previous  exanqile  used  two  impmtant 
assunqitions:  relative  stack  positicm  is  meaningful,  and  die  goal  state  desoiption  is  completely 
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specified.  Relaxing  these  limitations  will  allow  for  more  flexible  problem  solving  while  still 
retaining  linear-m'der  retrieval,  provided  a  storage  space  penalty  is  accepted. 

By  making  the  relative  stack  position  important,  the  first  assumption  restricts  the  al¬ 
lowable  block  configurations.  This  assumption  provides  for  a  very  compact  index  and  avoids 
storing  a  maao  operator  schenui  under  multiple  indices.  However,  if  stack  order  is  not  im- 
ptxtant,  a  different  ^proach  to  indexing  is  required.  A  single  index  cannot  accommodate 
all  permutations  of  a  configuration  when  there  are  two  or  more  stacks  of  the  same  height 
Therefore,  multiple  indices  are  required,  leading  to  a  space/time  trade-off;  either  a  macro 
operator  schema  must  be  stored  under  multiple  indices  during  learning,  or  multiple  indices 
must  be  generated  during  plantting,  causing  multiple  retrievals.  Assuming  retrieval  time  is 
to  be  minimized,  the  best  choice  is  to  store  macro  operator  schemas  under  multiple  indices 
during  learning;  with  this  method,  the  time  for  retrieval  is  unaffected  by  removing  the  first 
assun^ttion. 

The  second  assumption  requires  that  the  planner  be  provided  with  a  complete  goal 
(final)  state  description  rafiier  titan  a  set  of  goals  that  must  be  true  in  the  final  state.  Under 
the  indexing  method  described  so  far,  if  tins  assumption  were  to  be  relaxed,  then  retrieval 
would  require  that  indices  be  generated  for  all  possible  final  states  that  include  tiie  given  set 
of  problem  goals.  A  better  t^proach  is  to  modify  the  indexing  method  to  use  the  initial  state 
and  the  problem  goals  when  a  schema  is  learned,  rather  titan  the  initial  state  and  the  goal 
state.  These  problem  goals  become  the  primary  effects  of  the  learned  macro  operator.  A 
space  cost  could  be  incurred  if  the  problem  goals  are  not  minimally  specified  in  the  learning 
problems  (ie.,  side-effects  are  included  in  tiie  problem  descrqttions),  but  tins  has  no  inqtact 
on  retrieval  time.  The  usefulness  of  these  learning  situations  is  questionable,  however,  since 
maao  qtaata  schemas  are  retrieved  based  upon  exact  match  of  tiie  interface  characteristics 
(generalized  initial  stale  and  problem  goals/)>rimary  effects).  A  discipline  of  only  specifying 
the  problem  goals  would  prevent  storage  space  growth  and  provide  a  better  hit-rate  on  schema 
retrieval 
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4.43  Polynomial-Time  Retrieval.  A  particular  macro  operator  often  requires  only 
a  subset  of  the  initial  state  propositions  to  satisfy  its  preconditions.  Thus,  from  a  space 
standpoint,  it  is  attractive  to  reduce  redundancy  and  index  the  macro  operator  schema  on  its 
precmiditions,  rather  than  on  every  superset  of  the  preconditions  that  represents  an  initial  state. 
This  approach,  however,  produces  an  impedance  mismatch  that  precludes  linear-time  retrieval: 
a  problem  description  provides  complete  information,  but  the  macro  operator  schemas  are 
sttved  using  only  the  necessary  preconditions  under  which  they  were  learned.  Therefore,  a 
two-step  process  is  used  for  retrieval.  Indexing  is  based  only  on  goals  that  an  operator  schema 
achieves  (its  primary  effects).  To  retrieve  appropriate  reuse  candidates,  possible  candidates 
are  retrieved  based  on  the  goals  they  achieve  and  then  die  candidates  are  tested  to  see  if  all 
dieir  preconditions  can  be  established  by  the  problem  initial  state.  Only  those  passing  the  test 
are  forwarded  to  the  planner. 

When  a  maao  qierator  schema  is  learned,  the  stmage  index  is  based  only  on  its  primary 
effects.  For  retrieval,  the  problem  goals  are  used  to  form  the  index;  dierefore,  there  is  no 
iai|>edance  mismatch  and  the  retrieval  time  is,  at  worst,  linear  with  ihe  size  of  the  library.  All 
schemas  that  achieve  those  goals  will  be  retrieved;  let  the  number  of  retrieved  schemas  be  M. 
Eadi  schema  wUl  have  a  set  of  preconditions;  let  schema  i  have  iV,  preconditio  mally,  the 

initial  state  of  a  problem  has  L  propositions;  the  set  of  Ni  preconditions  may  or  may  not  be  a 
subset  of  the  initial  state.  When  instantiated,  a  macro  operator  schema’s  A',  preconditions  can 
be  established  if  aU  preconditions  are  a  subset  of  the  L  propositions  of  the  initial  state.  Each 
prectmdition  of  each  schema  must  be  tested  against  the  problem  initial  state,  resulting  in  a 
maximum  of  L*  Ni  comparisons.  Since  Ni  cannot  be  greater  than  L  for  an  appropriate 
schema,  the  number  of  tests  is  upper-bounded  by  M  *  L^.  Therefore  maao  testing  requires 
time  of  orda  0{n^).  Because  the  hash  table  lookup  requires,  at  worst,  0{M)  time,  the  time 
conq>lexity  of  finding  appropriate  maao  opaator  schemas  is  a  polynomial  of  order  0{n^). 

SectUHi  5.3  describes  an  experiment  in  which  IQNGE  solves  problems  by  selective 
reuse  using  ttie  0{n^)  time  retrieval  method  discussed  hae.  In  the  experiment,  file  number 
of  maao  tqiaata  schema  preconditions  and  fiie  number  of  initial  state  propositions  are  fixed 
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while  the  number  of  macro  operator  schemas  increases.  The  results  are  consistent  with  the 
analysis  presented  here;  in  the  worst  case,  increasing  the  library  size  causes  a  performance 
degradation  that  is  linear  in  the  size  of  the  library. 

Analyzing  Selective  Reuse 

There  are  alternative  reuse  policies  that  could  be  followed  in  a  macro  planner.  However, 
to  contain  the  utility  problem,  selective  reuse  (or  the  relaxed  form  of  selective  reuse)  is 
the  best  policy  because  it  most  strictly  limits  search  and  contains  search-space  expansicHi. 
Furdiermore,  if  reuse  candidates  are  guaranteed  to  be  ^propriate  (solve  the  whole  problem), 
then  the  planner  can  consider  just  one  of  them,  or,  by  extension,  if  reuse  candidates  are  likely 
to  be  ^propriate,  then  reuse  will  probably  succeed  if  die  planner  limits  consideration  to  a 
few  of  them.  By  considering  only  one  a  few  of  die  reuse  candidates,  the  size  of  the  search 
space  is  constrained  further.  These  are  the  fundamental  ideas  of  selective  reuse,  a  reuse  policy 
designed  to  contain  the  utility  problem  in  macro  planning. 

Alternatives  to  reuse  candidates  that  solve  the  whole  problem  are: 

1.  macros  whose  effects  can  achieve  all  problem  goals  but  whose  preconditions  cannot  all 
be  established  by  existing  plan  operators, 

2.  macros  whose  preconditions  can  all  be  established  by  existing  plan  operators  but  whose 
effects  cannot  achieve  all  problem  goals,  and 

3.  macros  whose  preconditions  cannot  all  be  established  by  existing  plan  operators  and 
whose  effects  cannot  not  achieve  aU  problem  goals. 

dbnsidering  any  or  all  of  diese  reuse  candidates  in  addition  to  those  that  solve  the  whole 
problem  expands  the  search  space  being  traversed  by  the  planner  because  there  are  more 
choices  available.  Furthermore,  all  diree  alternatives  require  additional  search,  while  reuse 
candidates  that  solve  die  whole  problem  do  not  Inserting  a  macro  of  the  first  type  adds 
goals  for  prectmdition  establishment  to  die  set  of  outstanding  goals  being  considered  by  the 
planner.  Hiese  new  goals  will  also  have  to  be  achieved  through  search,  and  the  amoimt  of 
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search  necessary  to  achieve  them  is  unknown  and  unconstrained.  Inserting  a  macro  of  the 
second  type  means  that  the  choice  may  have  to  be  retracted,  resulting  in  backtracking,  lost 
search  effort,  and  additionai  search.  For  example,  if  stack(A^)  were  a  macro  for  Sussman’s 
Anomaly  (see  Figure  15  on  page  53  and  Figure  7  on  page  20),  inserting  it  would  not  lead  to 
a  solution  and  backtracking  would  result  Inserting  a  macro  of  the  third  type  would  require 
additional  search  for  both  reasons.  Because  considering  any  of  these  alternatives  in  addition 
to  whole-problem  solutions  expands  the  search  space  more  and  causes  additional  search,  any 
policy  that  considers  them  is  not  as  useful  for  limiting  the  utility  problem  as  selective  reuse. 

4.6  Solving  Pmblems  Incrementally  with  Relaxed  Selective  Reuse 

Section  4.4  described  an  example  that  depended  on  finding  a  whole-problem  solution 
(a  macro  that  is  guaranteed  to  be  appropriate  and  also  solves  the  whole  problem).  Often,  a 
whole-problem  solution  does  not  exist  in  the  library  of  reuse  candidates.  If  so,  reuse  may  still 
be  possible  by  decomposing  the  problem  and  finding  a  set  of  ^propriate  macros  that  solve 
pieces  of  it  To  test  for  macro  ^propriateness,  however,  the  plaimer  itself  must  generally  be 
used  to  solve  the  problent  making  the  test  as  expensive  as  solving  the  planning  problem  itself. 
A  less-expensive  strategy  is  to  find  macros  that  are  very  likely  to  be  appropriate.  For  example, 
perhtq)s  it  is  possible  to  identify  parts  of  the  problem  which  are  likely  to  be  ind^>eadent  If 
so,  dien  macros  which  are  used  to  solve  these  subproblems  will  probably  not  be  clobberers 
of  each  ofiier’s  establishments,  and  using  them  is  more  likely  to  result  in  a  solution.  Under 
these  assunq)tions,  the  macros  found  using  the  retrieval  method  presented  in  Section  4.4.3  are 
likely,  rather  than  guaranteed,  to  be  tqjpropriate  because  sub-problem  solutions  may  fail  to 
integrate.  This  is  the  basis  of  the  relaxed  form  of  selective  reuse. 

Selective  Reuse  (relaxed  form):  The  policy  of  considering  only  a  few  reuse 
candidates  which  are  appropriate  for  solving  sub-problems  of  a  problem  for  which 
no  whole-problem  solutimi  can  be  found. 

Solving  a  sub-problem  inaeases  the  number  of  propositions  that  possibly  match  macro 
qperator  precrniditions  when  testing  for  ^plicability,  but  retrieval  still  requires  0{n^)  time. 
As  described  in  Section  4.4.3,  each  learned  macro  operator  schema  is  stored  using  an  index 
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based  only  on  its  primary  effects.  There  is  no  change  in  library  retrieval  when  hashing  is 
used:  the  problem  goals  are  used  to  form  the  index;  therefore,  there  is  no  impedance  mismatch 
and  the  retrieval  time  is,  at  worst,  linear  widi  the  size  of  the  library.  As  before,  all  schemas 
that  achieve  those  goals  will  be  retrieved;  let  the  number  of  retrieved  schemas  be  M.  Each 
schema  will  have  a  set  of  preconditions;  let  schema  i  have  Ni  preconditions.  As  before, 
the  initial  state  has  L  propositions,  the  set  of  which  may  or  may  not  be  a  superset  of  the  iV, 
preconditions.  However,  because  odier  plan  operators  may  be  able  to  establish  the  macro’s 
precmiditions,  the  effects  of  these  operators  must  be  added  to  the  initial  state  propositions 
when  testing  to  see  if  the  macro  preconditions  can  be  established  widiout  additional  work. 
Suppose  there  are  K  effects  corresponding  to  odier  plan  operators  that  may  be  able  to  establish 
die  macro’s  preomditions.  When  instantiated,  a  macro  operator  schema’s  A,  preconditions 
can  be  established  if  they  are  a  subset  of  the  L  propositions  of  the  initial  state  unitmed  with  the 
K  effects  of  possibly'before  (^lerators.  Suppose  diere  are  J  of  these  propositions  and  effects, 
where  J  >  L  always.  Substituting  J  for  Z  in  die  argument  made  in  Section  4.4.3  shows 
that  the  time  complexity  of  this  ^proach  to  maao  operator  schema  retrieval  is,  at  worst,  a 
procedure  whose  time  complexity  is  0(n®),  as  before. 

4.7  The  Impact  of  Efficient  Retrieval  and  Selective  Reuse  on  the  Utility  Problem 

Efficient  retrieval  and  selective  reuse  are  effective  strategies  for  stricdy  limiting  per¬ 
formance  degradation  diat  possibly  acconqianies  reuse,  but  these  methods  do  not  eliminate 
the  utility  problem.^  Whenever  an  appropriate  reuse  candidate  does  not  exist,  the  effort  spent 
looking  for  an  sqiprc^riate  candidate  will  be  wasted  and  macro  planning  performance  will  be 
worse  than  generative  planning.  Efficient  retrieval  and  a  policy  of  selective  reuse  are  designed 
to  minimize  the  effmt  spent  looking  for  an  appropriate  reuse  candidate.  These  methods  and 
the  methods  diat  promote  boieficial  reuse  discussed  in  Ch^ter  m  combine  to  contain  the 
utili^  problem  and  lessoi  its  inqxHtance. 

'^nie  same  is  tnie  for  sdecdve  teaming  and  case-based  planning — diese  methods  do  not  eliminate  the  utility 
inoUemeidier. 
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4.8  Summary 

After  analyzing  the  utility  problem,  this  cht^ter  described  two  mediods  for  containing 
the  utility  problem  in  macro  planning.  First,  it  introduced  the  policy  of  selective  reuse  to  control 
search  and  contain  die  expansion  of  the  search  space  caused  by  considering  mwe  qierators. 
Selective  reuse  causes  die  planner  to  consider  only  whole-problem  solutions  (tqipropriate 
macros)  or,  in  its  relaxed  form,  whole-problem  solutions  to  sub-problems  (likely-appropriate 
nuux'os).  Second,  it  presented  a  generally-appUcable  retrieval  method  for  macro  planning 
that  finds  an  appropriate  or  likely-appropiiate  reuse  candidate  in  0{n^)  time  when  complete 
problem  information  exists.  The  perfcnmance  of  this  retrieval  mediod  is  independent  of  the 
size  of  die  macros  in  die  library  and  varies  only  linearly  widi  the  size  of  the  library.  Selective 
reuse  was  shown  to  better  contain  search  and  search-space  expansion  bett^  dian  alternative 
reuse  strategies.  A  relaxed  form  of  selective  reuse  was  defined  u>  allow  reuse  when  no  whole- 
problem  solutitxis  exist  in  the  library  of  reuse  candidates.  Using  these  mediods,  HINGE 
effectively  contains  the  utility  problem  without  selective  learning.  The  next  ch^ter  presents 
eiqierimental  results  diat  are  ctuisistent  with  mialytica'  arguments  made  in  this  chapter. 
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V;  Empirical  Results 


5.1  Introduction 

In  Ch^tm  m  and  IV  established  die  computational  advantages  of  the  planning,  re¬ 
trieval.  and  reuse  t^proaches  developed  for  containing  die  utility  problem.  This  cheater 
describes  die  con^utational  results  observed  when  these  ^proaches  were  implemented  in 
HINGE.  In  particular,  this  chapter  concentrates  on  HINGE’s  performance  when  retrieving 
reuse  candidates  from  large  libraries,  when  using  an  alternative  reuse  policy,  and  when  learn¬ 
ing  from  random  problems. 

5.2  The  Domain  and  Method  Used 

Given  the  (^ator  representation  used  in  this  research  and  defined  in  Section  2.2.2, 
any  domain  diat  can  be  represented  by  HINGE  operators  could  have  been  chosen  to  gain 
conqiutational  experience  with  HINGE.  I  diose  to  use  die  block-stacking  domain  for  this 
purpose  because  block-stacking  problems  are  familiar  to  many  researchers  and  are  simple  to 
understand.  The  block-stacking  domain  is  described  in  (Norvig.  1992:136-142). 

To  avoid  bias  in  choosing  a  test  set,  I  used  (xily  randomly-generated  prrdilems.  I 
developed  a  random-problem  g^eradx  diat  works  by  producing  two  states,  using  USP’s 
pseudo-random  number  generate  to  decide  die  placement  of  each  block  in  each  state.  The 
random-problem  generate  selects  the  sinqiter  state  (the  one  with  the  greater  numbo’  of  smalls 
block  stacks)  as  the  problem’s  initial  state.  The  problem  goals  are  produced  by  randomly 
selecting  prt^iositions  from  the  remaining  state,  which  represents  one  of  the  possibly  many 
goal  states. 

I  measured  perfrxmance  in  toms  of  die  time  HINGE  required  to  find  a  plan  under 
different  conditums  for  22  randomly-generated  6-block  block-stacking  problems.  Planning 
time  is  rqxxied  eidier  fcv  each  individual  problem  ot  as  a  mean  planning  time  fm  all  problems 
in  the  test  set  A  Sun  IPC  was  used  fOT  some  measurements  while  odiers  were  made  using  a  Sun 
SPARCstatum  10.  Iherefore,  absolute  planning  times  vary  because  of  different  processing 
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power  of  die  different  con^uters  used.  However,  direct  con^arisons  were  only  made  between 
problem  runs  on  the  same  processor. 

5  J  The  Effect  of  Large  Libraries 

To  examine  the  worst-case  performance  of  HINGE’s  retrieval  mediod  as  a  function 
of  the  size  of  the  library.  HINGE  was  repeatedly  trained  on  the  test  set  of  22  problems  and 
perfmmance  with  reuse  was  measured  as  a  function  of  inaeasing  library  size.  The  polynomial 
retrieval  method  described  in  Section  4.4  was  used  at  all  times.  As  described  in  Chapter  IV. 
this  retrieval  mediod  uses  a  hash  table;  in  the  worst  case,  all  hash  tabte  ratries  are  stored  in  a 
list  indexed  by  a  single  hash  key.  To  retrieve  a  hash  entry  in  the  wmst  case,  each  elemrat  of 
die  list  must  be  examined,  an  qieradon  that  requires  time  that  is  a  linear  functitxi  of  die  length 
of  die  list  By  repeatedly  learning  solutions  to  the  santt  set  of  problems,  HINGE’S  worst-case 
library  is  built  and  HINGE’S  worst-case  retrieval  performance  can  be  measured. 

During  this  measuremoit.  HINGE  was  set  to  retrieve  all  ^prqiriate  reuse  candidates, 
but  only  oat  of  them  of  them  was  used^ ;  und^  these  ctmditions,  any  increase  in  planning  time 
is  due  to  die  retrieval  method  finding  addititmal  reuse  candidates.  Thus,  the  retrieval-method 
pafcHmance  was  measured  indirectly  by  measuring  planning  time. 

HINGE  was  initially  set  to  learn  macros,  but  not  allowed  to  reuse  them.  Instead, 
HINGE  solved  each  problem  in  die  test  set  by  generative  planning  using  only  the  three 
primitive  operatm  schemas  shown  in  Figure  IS  on  page  S3.  Aftv  solving  die  22  problems, 
the  operator  sdiema  library  contained  28  maao  r^ratm:  schemas^  as  well  as  the  (xiginal  3 
primitive  operator  sdiemas.  Next,  HINGE  was  set  to  reuse  these  maCTO  qp^atin’  sdiemas, 
but  not  to  team  new  ones.  Planning  times  w^  found  fOT  die  test  set  problems  white  HINGE 
used  this  small  library. 

^Only  one  tense  candidate  is  leqBired  to  solve  a  proWem  if  candidate  is  aptao|Mnatebecanse  it  must  be  a 

wlioleiKotdeffl  sdutkn. 

^The  kaaer  geneializes  and  states  all  significant  subproUetns,  so  each  i»oUem  resulted  in  learning  a  variaUe 
nnniber  of  inacro  opetater  schemas. 
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To  see  the  effect  of  increasing  the  library  size  on  retrieval  performance,  the  learning  filter 
that  ntvmally  prevents  redundant  macro  (^)erator  schemas  firom  being  stored  in  the  library  was 
disabled,  and  the  procedure  described  in  the  paragr^h  above  was  repeated  29  times.  After 
each  iteration,  the  same  28  maaos  were  learned  and  stored  in  the  library  using  die  same  set 
of  indices  derived  on  previous  iterations.  Thus,  after  five  iterations,  for  example,  a  particular 
macro  index  pointed  to  at  least  S  macro  operate  schemas.^  Normally,  HINGE  does  not  learn 
a  particular  plan  more  than  (Hice. 

Figure  32  shows  the  planning  time  required  for  generative  planning  to  solve  the  22 
problems  as  well  as  die  corresponding  solution  times  for  planning  widi  reuse  using  four 
different  sizes  of  die  macro  (^ator  schema  library.  Data  points  that  are  associated  with  a 
particular  set  of  amditums  are  omnected  by  a  line  segmoits  cm  the  graphs. 

PCX'  most  problems,  the  planning  time  increases  as  the  library  grows  because  die  retrieval 
method  must  test  additicxial  reuse  candidates.  The  planning  times  fw  problems  11  and  12 
did  not  vary  as  the  library  grew  because  die  solutions  for  these  problems  w^e  not  learned. 
These  two  problems  had  phanttxn  goals  whidt  required  planning  effort  to  assure,  but  the 
prciblem  sohiticms  did  not  involve  multqile,  related,  ccmcrete  c^attxs.  (See  Secticm  3.8.2  fcx 
a  desmpticm  of  the  filters  used  in  HINGE’S  learning  element)  As  shown  in  Hgure  32.  the 
increase  in  planning  time  caused  by  retrieval  from  largo-  libraries  varies  with  die  problem. 
Howevo,  f<x  each  {n-oblem,  die  time  required  to  plan  with  reuse  is  prqxirticMial  to  the  size  of 
the  macro  c^ator  sdiema  library.  Figure  33  shows  the  increase  in  the  mean  planning  time  as 
a  frmctiiCKi  of  library  size  fcx  eadi  of  diirty  iteratuxis.  Because  die  lllx-ary  size  was  increased  in 
die  worst  way,  these  results  illustrate  that  in  die  wcxst  case  and  fra-  the  test  set  used,  retrieval 
required  time  that  is  linear  in  the  size  of  the  library. 

^  A  particular  index  mi^  point  to  ame  than  5  macro  opetator  scfaemas  because  tbe  index  is  based  only  aa 
operator  sdiema  effects.  Some  of  tbe  22  problems  result  in  leaniing  macro  operator  schemas  diat  have  the  same 
effects  and  indices,  but  different  preconditions. 
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Figure  32.  Comparative  planning  times  widi  reuse  at  four  library  sizes. 
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Figure  33.  Average  planning  times  with  reuse  at  thirty  library  sizes. 
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Accwding  to  die  thetvetical  perftMtnance  derived  in  Section  4.4,  HINGE’S  retrieval 
method  is.  at  worst,  linear  in  die  size  of  the  library.  The  en^irical  results  presented  in  diis 
section  is  consistent  with  the  derived  dieory. 

5.4  Selective  Reuse  vs.  Macros  with  Abstract  Establishers 

As  discussed  in  Sectum  4.5,  selective  reuse  reduces  search  and  limits  expansimi  of 
the  search  space  expanskm  better  than  less  selective  reuse  policies.  One  exanqile  of  a  less 
selective  reuse  policy  is  a  policy  that  considers  reductions  whose  operate  preconditimis  cannot 
all  be  established  by  existing  plan  (^leratm^.  Such  preconditicms  must  be  established  by  an 
abstract  (^lerauv  in  the  reducticHi,  and  the  abstract  cqiarattx'  represents  future  wm'k  required 
of  die  planner.  Peifcxmance  data  were  collected  when  planning  with  reuse  and  allowing 
different  numbers  of  abstract-t^ieraticv-established  preconditioiis.  After  training  tm  the  test  set 
of  22  randomly-generated  problems,  HINGE  was  set  to  collect  planning  time  data  when  0, 1, 
2,  3,  and  4  abstract-tqierator-established  precmiditions  were  allowed  in  reduction  (^leratms. 
Selective  reuse  ctxresponds  to  allowing  0  abstract-<^alm-established  preconditkms. 

Figure  34  shows  that  die  average  planning  time  for  problems  in  the  test  set  increased 
as  the  allowed  number  of  abstract-t^ierator-established  preconditions  was  increased  from  0 
to  3.  The  planning  time  did  not  inarease  when  the  number  of  abstract-operator-established 
I»ec(mditions  was  increased  from  3  to  4  because,  for  die  test  set  used,  no  additional  reuse 
candidates  were  ctmsidered  as  aresult  of  dis  increase. 

As  described  in  Section  4.5,  selective  reuse  is  better  than  any  (rf die  tiuee  alternative  reuse 
policies  at  limiting  the  expansion  of  the  search  space  and  search  in  die  qiace.  hi  particular, 
sdective  reuse  reduces  search  more  dum  a  policy  that  considers  reductions  which  include 
abstract  (^lerators  because  die  latter  policy  requires  additimai  planning  work  to  establish 
preconditlcxis.  Ihe  results  shown  in  Figure  34  are  compatible  with  the  analysis  presented  in 
Section  4.5. 
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Hgure  34.  Perfwmance  as  a  function  of  the  allowed  number  of  new  operator  preconditions 
which  cannot  be  established  by  existing  plan  operatm-s. 

5  3  Learning  from  Random  Problems 

A  tenet  of  selective  learning  is  tiiat  die  utility  problem  makes  non-selective  learning 
on  random  problems  a  disastrous  course  of  actum.  However,  HINGE’S  efficient  retrieval 
and  selective  reuse  policy  omtain  the  costs  associated  with  die  utility  problem  and  lessen 
the  impact  of  having  learned  plans  diat  are  not  useful  fen:  die  current  planning  problem. 
PurdiemKue,  HINGE’S  planning  model  and  hierarchical  search  through  successively  fin^ 
deconqiosituHis  allows  flexible  insertitm  of  sinqile  plans  that  may  be  learned  from  randomly 
generated  problems. 

To  explo-e  non-selective  learning  in  HINGE  19  sets  of  10  randomly-generated  problems 
were  solved  and  their  solutions  learned.  After  learning  solutums  fru  each  set  of  randomly- 
generated  problems,  timing  data  were  collected  for  planning  with  reuse  cm  die  22  problems  of 
the  test  set  Figure  35  shows  diat  under  diese  conditums,  no  problem  in  the  test  set  required  a 
great  deal  of  extra  time  to  solve. 
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Figure  3S.  PerfcHmance  improvem^t  witfi  reuse  and  different  amounts  of  learning 
oppmiunities. 

HINGE  is  designed  to  control  the  utility  problem.  HINGE’S  attributes  have  been  shown 
analytically  to  contain  die  specific  costs  of  the  utility  problem  or  improve  the  likelihood  of 
reuse  and  its  accompanying  benefit  Allowing  HINGE  to  learn  190  random  problems  did  not 
drastically  intact  the  performance  of  any  erne  problem  in  the  test  set  a  result  that  is  in  accord 
wifii  die  previous  analysis  of  HINGE’s  features. 


VI.  Conclusions  and  Recommendations 

6.1  Conclusions 

This  research  describes  a  maao  planning  model  and  implementation  designed  to  contain 
the  utility  problem.  The  utility  problem  is  die  possibility  that  the  costs  of  reuse  sometimes 
exceed  its  benefit  In  this  research,  containing  the  utility  problem  depends  on  improving 
the  probability  of  reuse  and  strictly  limiting  the  costs  that  contribute  to  the  utility  problem. 
Because  HINGE  is  based  on  a  plan-space  planning  model,  it  can  solve  a  wider  variety 
of  planning  problems  with  reuse  than  state-space  macro  planners  can.  Furfiiermore.  unlike 
typical  case-based  planners.  HINGE  and  other  macro  planners  are  more  flexible  in  terms  of  the 
number  and  granularity  of  plans  that  are  reused.  HINGE  has  a  unique  method  of  hierarchical 
search  for  reuse  candidates  whidi  is  supported  by  decomposition  methods  and  inaeases  die 
probability  of  reuse.  To  ctmstrain  costs  which  contribute  to  the  utility  problem,  an  efficient 
retrieval  mediod  was  developed  to  reduce  die  retrieval  cost  associated  with  die  utility  problem. 
By  design,  this  retrieval  mediod  requires  time  diat  varies  linearly  with  three  parameters:  the 
number  of  reuse  candidates  in  a  library,  die  number  of  preconditions  for  maaos  resulting 
from  retrieval  of  a  reuse  candidate,  and  the  number  of  propositicHis  in  the  initial  state.  A 
policy  of  selective  reuse  was  develqied  to  reduce  search  and  limit  the  search-space  eiqianskm 
caused  by  cmisidering  reuse  candidates  in  additicm  to  primitive  cqieratms.  Flexible  maao 
insertkm,  hierardiical  search  providing  multqile  oppmiunities  for  reuse,  efficient  retrieval, 
and  selective  reuse  combine  to  improve  perfOTmance  in  HINGE.  Together,  these  mediods 
lessen  die  inqiact  of  die  utility  problem  in  macro  planning  widiout  requiring  selective  learning 
ami  its  diaracteristic  potential  for  ignoring  a  plan  with  future  value. 

62  Specific  Contributions 

The  novel  planning  ideas  and  cmitribudons  of  this  research  are: 

1.  A  model  of  planning  and  reuse  that  explicitly  represents  features  of  desirable  macro 
operators  and  supports  more  flexible  insertion  of  macro  operators  into  the  developing 
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plan.  the  planning  model  I  developed,  abstract  operators  represent  goals  which  are 
used  to  index  macro  operatcvs  that  achieve  them.  This  ability  to  represent  goals  which 
should  be  achieved  simultaneously  by  a  macro  operator  is  extremely  useful  for  search 
control.  When  a  good  macro  operator  is  retrieved,  HINGE’S  plan-space  planning  model 
allows  it  to  be  inserted  anywhere  in  die  plan.  This  feature  allows  reuse  that  is  precluded 
by  state-space  planning  models  and  case-based  planning  models. 

2.  A  unique  hierarchically-structured  search  control  that  encourages  subproblem  learning 
and  efficient  reuse  while  ensuring  completeness.  HINGE  searches  for  die  most  pow¬ 
erful  reuse  candidates  first,  using  a  hierarchically-structured  search  mechanism  where 
abstraction  levels  are  based  (xi  the  number  of  goals  that  HINGE  attempts  to  solve  si¬ 
multaneously.  This  hierarchical  search  is  supported  by  several  decomposition  methods 
that  allow  abstract  operator  reduction  when  no  impropriate  or  Ukely-^propriate  reuse 
candidate  can  be  found.  HINGE  does  not  differentiate  between  macros  and  primitive 
operators.  The  effect  of  diese  choices  during  planning  is  to  facilitate  opportunistic 
reuse  of  whole-problem  solutions  and  flexible  insertion  of  subproblem  solutions  when 
decomposition  is  necessary.  If  reuse  fails.  HINGE  solves  the  problem  using  only  prim¬ 
itive  operatCHS  and  a  conmlete  search  strategy,  meaning  that  HINGE  always  finds  a 
solution  to  a  planning  problem  if  a  sohtticm  exists.  When  learning,  any  sub-parts  of  the 
hierarchical  search  tree  that  ctxrespond  to  useful  knowledge  are  learned  by  HINGE’s 
learning  comptment 

3.  A  polynomial-order  procedure for  retrieving  only  applicable  macros,  despite  their  size  or 
number.  An  important  result  of  this  research  is  the  realization  that,  if  complete  problem 
informatkxi  exists,  it  is  possible  to  develop  a  discriminating  index  which  can  be  used 
to  efficiently  retrieve  only  tpprtmriate  concq>ts  learned  by  EBL  (or  any  other  learning 
technique),  de^te  the  size  or  number  of  the  c(nicq)ts  learned.  If  a  discriminating 
index  is  used  and  a  whole-problem  solution  exists  in  the  library  of  learned  ctmcepts. 
dien  retrieval  costs  that  contribute  to  the  utility  problem  can  be  less  than  the  cost  of 
^neral  search  and  matdiing.  For  HINGE,  this  seardi  requires  O(n^)  time,  while 
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the  search  corresponding  to  the  planning  itself  requires  exponential-order  time.  If  no 
whole-problem  solution  exists  in  the  library,  these  same  results  apply  to  sub-problem 
solutions  except  that  the  retrieved  solutions  are  likely,  rather  than  guaranteed,  to  be 
^propriate. 

4.  A  policy  of  selective  reuse  that  limits  the  number  of  macros  considered  and  restricts  the 
future  work  they  entail.  If  macros  that  do  not  solve  the  whole  problem  are  considered, 
the  search  space  is  expanded  more  than  if  only  whole-problem  solutions  are  considered. 
Whole  problem  solutions  require  no  additional  search,  while  the  alternatives  require 
an  unpredictable  amount  of  search.  Furthermore,  if  multiple  whole-problem  solutions 
exist,  there  is  no  need  to  consider  more  dian  one  of  them,  or,  by  extension,  if  multiple 
reuse  candidates  which  are  likely  to  be  whole-problem  solutions  exist,  then  considering 
only  one  or  a  few  of  diem  is  likely  to  lead  to  a  solution.  Limiting  the  number  of  reuse 
candidates  considered  is  an  effective  mediod  of  limiting  the  search  space.  Therefore, 
in  selective  reuse,  just  one  whole-problem  solution  is  considered,  or  if  appropriateness 
cannot  be  guaranteed,  then  a  relaxed  fcxm  of  selective  reuse  only  considers  a  few  reuse 
candidates  diat  are  likely  to  be  appropriate  (for  sub-problems).  For  domain-independent 
planning,  any  other  policy  re-introduces  costs  associated  with  the  utility  problem. 

6J  Recommended  Future  Work 

There  are  three  int^esting  extensions  of  this  research,  all  involving  search  control  The 
first  extension  involves  inserting  partial  domain  knowledge  using  abstract  operators.  The 
second  idea  is  to  find  new  ways  to  use  the  library  as  a  source  of  knowledge  for  finding 
decompositions.  The  dfird  idea  is  to  focus  attention  in  planning  by  raacapsulating  parts  of  the 
plan  when  the  details  of  those  parts  are  not  likely  to  be  useful  for  establishing  preconditicHis 
of  new  (q)erat(»s. 

Abstr»:t  operators  introduced  in  Chapter  m  provide  a  means  for  introducing  partial 
domain  knowledge  into  a  plan  when  it  is  available.  The  method  for  doing  so  is  to  add  known 
{Mrectxiditicms  and  effects  to  abstract  operators.  When  preconditions  are  added  to  abstract 


operators,  diese  preconditions  are  established  sooner  during  planning  than  they  are  when 
they  (mly  appear  in  die  reduction  of  die  abstract  operator.  Known  effects  may  also  be  added 
to  id^tify  known  side-effects  that  accompany  the  goals  represented  by  a  HINGE  abstract 
operator.  When  known  effects  are  added  to  an  abstract  operator,  diese  effects  may  be  used 
in  establishments  sooner  or  may  give  rise  to  additional  ordering  constraints  sooner  than  if  the 
effects  do  not  ^pear  until  die  abstract  operator  is  reduced. 

Using  the  library  as  a  source  of  knowledge  for  finding  deconqiositions  appears  to  be 
fruitful,  but  more  research  is  required.  New  indexing  methods  are  potentially  important  for 
using  the  library  in  this  way.  More  advanced  techniques  for  learning  appropriate  decomposi¬ 
tions  from  past  problem  solving  are  possible:  diese  techniques  probably  require  a  redefinition 
of  die  library  contents.  Either  of  these  extensions  can  co-exist  with  HINGE’S  current  library 
and  retrieval  procedures. 

Planning  requires  0(a”)  time  for  a  plan  of  n  operators  (Bylander,  1991).  If  the  number 
(and  conqilexity)  of  operators  in  a  plan  can  be  reduced  on-die-fly,  planning  can  be  more 
efficient  One  way  to  reduce  the  number  and  complexity  of  curators  in  a  plan  is  to  encapsulate 
part  of  the  plan  in  a  macro  qierator  and  to  replace  that  part  with  die  macro.  This  enc^suladon 
^proach  is  effective  whm  establishments  for  preconditions  of  new  operators  can  be  found  in 
the  macro  effects  or  in  the  effects  of  other  plan  operators,  but  do  not  depend  on  the  primitive 
opaators  encapsulated  by  the  macro.  The  approach  is  likely  to  be  effective  when  the  part 
of  the  plan  chosen  for  oicapsuladon  is  likely  to  be  independent  from  die  other  parts  of  the 
plan.  The  cooqilexity  of  maCTOs  that  arise  from  enctqisulation,  in  terms  of  the  number  of 
IHOcondittons  and  effects,  is  sometinies  much  greater  than  for  odio*  operattars,  but  in  the  case 
diat  many  establishments  are  made  internally,  macro  conqilexity  differs  litde  from  that  of  other 
operators.  Thotefore,  pieces  of  the  developing  plan  that  are  likely  to  be  independent  and  have 
many  internal  establishments  are  good  candidates  for  replacement  with  a  macro.  It  is  worth 
mentioning  diat  a  particular  operator,  having  failed  to  find  establishments,  might  view  such  a 
macro  in  its  eiqianded  frnm  to  try  to  find  an  establishment,  but  diis  does  not  mean  that  other 
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operators  need  to  do  so.  Encapsulation  of  parts  of  the  plan  with  macros  appears  promising  as 
a  method  of  focusing  attention  during  generative  planning. 
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